Thursday, July 30, 2015

How I resolved Network Adapter(WIFI) issue after Window 10 Upgrade

I was one of the lucky ones like you to upgrade my laptop with the official Windows 10 on the day of release (07/29).

A little icon on the system tray popped up with the message Windows 10 is ready to install when I turned on my laptop in the morning. I wanted to give a try so I went ahead and followed the on screen instruction for the upgrade. The upgrade was very smooth. I didn't even have to take backup. The upgrade utility took care of everything to keep all my personal data and as well the applications installed already.

I was happy to see the new start menu and features in new OS. Even more happy to see  all my data after first login. My happiness didn't last longer. When I opened chrome to see how things over the internet. I got the "Unable to connect to Internet".  Oh! did I forget to connect to my wifi? Oh No.. I don't see any wifi access points? Oh Hell No.. I don't have an option to set up wireless network.

I was all over the internet looking for solution. Updated the driver quite few times and nothing worked. With the tiring and sad mood, I wanted to get rid of new OS (Windows 10). Luckily Windows 10 had an option to downgrade to previous version of OS i.e. Windows 8.1(The options is available in the Upgrade and Security) section. (Downgrade Windows 10).

Downgrade from windows 10 to 8.1 was smooth again. To my surprise my wifi network configured automatically post downgrade. Voila... This time my internet connection worked in Windows 8.1.

My ego pushed me to upgrade to Windows 10 again. But this time I did little bit home work and research and listed down the possible roadblocks for the network issue in Windows 10.
I came up with the following list
  • Firewall
  • VPN (Virtual Private Network)

Unfortunately I had installed the clients for both firewall (Symantec antivirus) and VPN (Cisco VPN Client). I uninstalled these softwares before attempting to upgrade again. I restarted the windows 8.1 after uninstalling the antivirus and VPN client and upgraded to Windows 10. To my surprise this time I got the wireless network configured automatically and I am connected to my default WIFI router.
Everything works like charm...... I reinstalled the antivirus in Windows 10 and no issues.

Tuesday, March 10, 2015

Download attachments from SOAP web service- streaming of contents

SOAP Client for Downloading Attachments - Streaming

We have seen uploading of larger data files using SOAP with attachments in the previous post(Upload attachments using SOAP). We will see how to download/stream larger files from SOAP web service.


Java Mail API Problem
The SOAP web service client for downloading attachments we will be discussing uses the MimeMessage structure but we will avoid using the JAVA MAIL package as it is not efficient while downloading the contents. If you use MimeMessage java api to get the MimeBodyPart then you'll be in trouble as MimeMessage loads the entire content as byte array into memory for calculating the size and number of MimeBodyParts. So we will avoid using it for larger files if you have small JVM memory.

How to stream larger files
The memory intensive operation can be avoided using our own custom code to parse the response from the SOAP web service. 

The following example shows the SOAP message response with the attachments. This example is for Oracle UCM's GetFileByName operation. 

The SOAP response message package is constructed using the  Multipart/Related  media type. The structure of the response stream is as follows:
  1. The primary SOAP message must be carried in the root body part of the Multipart/Related structure. Consequently the type parameter of the Multipart/Related media header will always equal the Content-Type header for the primary SOAP 1.1 message, i.e., text/xml.
  2. Referenced MIME parts must contain either a Content-ID MIME header.
Content-Type: multipart/related; type="text/xml"; start="rootContent"; boundary="----=_Part_12_21424854.1424891476173"
SOAPAction: "http://www.stellent.com/CheckIn/"
Content-Length: 1818


------=_Part_12_21424854.1424891476173
Content-Type: text/xml; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-ID: rootContent

<soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:get=\"http://www.stellent.com/GetFile/\">
   <soapenv:Header/>
   <soapenv:Body>
      <get:GetFileByName>
         <get:dDocName>Test.txt</get:dDocName>    <get:revisionSelectionMethod>LatestReleased</get:revisionSelectionMethod>
         <get:rendition>primary</get:rendition>
         <get:extraProps>
            <get:property>
               <get:name>soapResponseType</get:name>
               <get:value>Multipart/Related</get:value>
            </get:property>
         </get:extraProps>
      </get:GetFileByName>
   </soapenv:Body>
</soapenv:Envelope>
------=_Part_12_21424854.1424891476173
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
Content-ID: test.log

Hello

------=_Part_12_21424854.1424891476173--

As you can see above the response stream starts with boundary (------=_Part_12_21424854.1424891476173). Each MimeBodyParts are separated by boundary. The content of each MimeBodyPart starts after blank line. The blank line is used to seperate the headers and content of each MimeBodyPart. Using apache LineInputStream we read each line until we encounter the blank line for the attachment content. Once the blank line preceeding the attachment body is read we will switch back to BufferInputStream for efficiency. Don't forget to ignore the boundary line which indicates the end of response and exclude it from writing into the file.


   1 
import java.io.BufferedInputStream;
   2 
import java.io.BufferedOutputStream;
   3 
import java.io.FileOutputStream;
   4 
import java.io.IOException;
   5 
import java.io.InputStream;
   6 
import java.util.Properties;
   7 
 
   8 
import javax.activation.DataHandler;
   9 
import javax.mail.Session;
  10 
import javax.mail.internet.MimeBodyPart;
  11 
import javax.mail.internet.MimeMessage;
  12 
import javax.mail.internet.MimeMultipart;
  13 
import javax.mail.internet.PreencodedMimeBodyPart;
  14 
 
  15 
import org.apache.axis.utils.StringUtils;
  16 
import org.apache.commons.httpclient.Header;
  17 
import org.apache.commons.httpclient.HeaderElement;
  18 
import org.apache.commons.httpclient.HttpClient;
  19 
import org.apache.commons.httpclient.HttpException;
  20 
import org.apache.commons.httpclient.HttpMethod;
  21 
import org.apache.commons.httpclient.NameValuePair;
  22 
import org.apache.commons.httpclient.methods.PostMethod;
  23 
 
  24 
import com.sun.mail.util.LineInputStream;
  25 
 
  26 
public class UCMDownloader {
  27 
 
  28 
        public static void main(String[] args) {
  29 
                HttpClient client = new HttpClient();
  30 
                PostMethod method = new PostMethod(
  31 
                                "http://www.oracle.com:16200/_dav/cs/idcplg");
  32 
                long start = System.currentTimeMillis();
  33 
                String fileName = "Test.txt";
  34 
 
  35 
                try {
  36 
 
  37 
                        method.addRequestHeader("SOAPAction",
  38 
                                        "\"http://www.stellent.com/GetFile/\"");
  39 
 
  40 
                        MimeMessage message = new MimeMessage(
  41 
                                        Session.getDefaultInstance(new Properties()));
  42 
                        MimeMultipart mp = new MimeMultipart();
  43 
 
  44 
                        // Create the first body part
  45 
                        StringBuffer requestXML = constructRequest(fileName);
  46 
                        long requestSize = requestXML.length();
  47 
                        MimeBodyPart rootPart = new PreencodedMimeBodyPart("8bit");
  48 
                        rootPart.setContentID("rootContent");
  49 
                        rootPart.setDataHandler(new DataHandler(new RequestXmlDataSource(
  50 
                                        requestXML.toString())));
  51 
 
  52 
                        mp.addBodyPart(rootPart, 0);
  53 
 
  54 
                        message.setContent(mp);
  55 
                        message.saveChanges();
  56 
 
  57 
                        String cType = message.getHeader("Content-Type")[0];
  58 
                        int idx = cType.indexOf("boundary");
  59 
                        int boundryLength = 0;
  60 
                        if (idx != -1) {
  61 
                                boundryLength = cType.substring(idx + 9).length();
  62 
                        }
  63 
 
  64 
                        long contentLength = (boundryLength * 2) + 2 + requestSize + 6;
  65 
 
  66 
 
  67 
                        method.setDoAuthentication(true);
  68 
                        method.setRequestBody(requestXML.toString());
  69 
 
  70 
                        method.addRequestHeader("Content-Length",
  71 
                                        String.valueOf(requestXML.length()));
  72 
                        method.addRequestHeader("Content-Type", "text/xml;charset=UTF-8");
  73 
                        method.getParams().setParameter("http.socket.timeout",
  74 
                                        new Integer(0));
  75 
 
  76 
                        // send the request
  77 
                        int status = client.executeMethod(method);               
  78 
                       
  79 
                        // process the response
  80 
                        contentLength = getContentLength(method);
  81 
                       
  82 
                        String boundary = getBoundary(method);
  83 
                        if (boundary != null) {
  84 
                                // Boundary foe each Body Part
  85 
                                boundary = "--" + boundary;
  86 
                        }
  87 
 
  88 
                        BufferedInputStream bin = new BufferedInputStream(
  89 
                                        method.getResponseBodyAsStream());
  90 
                        LineInputStream lin = new LineInputStream(bin);
  91 
 
  92 
                        boolean rootBoundaryEncountered = false;
  93 
                        boolean attachmentBoundaryEncountered = false;
  94 
 
  95 
                        String line = null;
  96 
                        long readSoFar = 0;
  97 
                        while ((line = lin.readLine()) != null) {
  98 
                                readSoFar += line.length() +2; // 2 for \r\n
  99 
                                if (line.equals(boundary)) {
 100 
                                        if (!rootBoundaryEncountered) {
 101 
                                                // boundary for root body part
 102 
                                                rootBoundaryEncountered = true;
 103 
                                                continue;
 104 
                                        } else { // root bodypart already read so we are ready to
 105 
                                                                // read next body part
 106 
                                                attachmentBoundaryEncountered = true;
 107 
                                        }
 108 
                                } else if (line.length() == boundary.length() + 2
 109 
                                                && line.startsWith(boundary) && line.endsWith("--")) {
 110 
                                                // End of response
 111 
                                        line = null;
 112 
                                        break;
 113 
                                } else if (!StringUtils.isEmpty(line)) {
 114 
                                        System.out.println(line);
 115 
                                        // header part and SOAP XML goes here
 116 
                                } else { // the attachment content always follows the blank line
 117 
                                        if (attachmentBoundaryEncountered) {
 118 
                                                // save the attachment content into file
 119 
                                                saveAttachment(bin, boundary, contentLength, readSoFar);
 120 
                                                break;
 121 
                                        }
 122 
                                }
 123 
                        }
 124 
 
 125 
                } catch (HttpException e) {
 126 
                        e.printStackTrace();
 127 
                } catch (IOException e) {
 128 
                        e.printStackTrace();
 129 
                } catch (Exception e) {
 130 
                        e.printStackTrace();
 131 
                } finally {
 132 
                        long end = System.currentTimeMillis();
 133 
                        System.out.println("Time taken for [" + fileName + "]: "
 134 
                                        + (end - start));
 135 
                        // release any connection resources used by the method
 136 
                        method.releaseConnection();
 137 
                }
 138 
        }
 139 
 
 140 
        public static StringBuffer constructRequest(String fileName) {
 141 
                StringBuffer request = new StringBuffer();
 142 
                request.append("<soapenv:Envelope
 143 
                        xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\"
 144 
                        xmlns:get=\"http://www.stellent.com/GetFile/\">\r\n"
 145 
                                + "   <soapenv:Header/>\r\n"
 146 
                                + "   <soapenv:Body>\r\n"
 147 
                                + "      <get:GetFileByName>\r\n"
 148 
                                + "         <get:dDocName>"+ fileName +"</get:dDocName>\r\n"
 149 
                                + "         <get:revisionSelectionMethod>
 150 
                                LatestReleased</get:revisionSelectionMethod>\r\n"
 151 
                                + "         <get:rendition>primary</get:rendition>\r\n"
 152 
                                + "         <get:extraProps>\r\n"
 153 
                                + "            <get:property>\r\n"
 154 
                                + "               <get:name>soapResponseType</get:name>\r\n"
 155 
                                + "               <get:value>Multipart/Related</get:value>\r\n"
 156 
                                + "            </get:property>\r\n"
 157 
                                + "         </get:extraProps>\r\n"
 158 
                                + "      </get:GetFileByName>\r\n"
 159 
                                + "   </soapenv:Body>\r\n"
 160 
                                + "</soapenv:Envelope>");
 161 
 
 162 
                return request;
 163 
        }
 164 
 
 165 
        public static String getBoundary(HttpMethod httpMethod) {
 166 
                Header h = null;
 167 
                String boundary = null;
 168 
 
 169 
                h = httpMethod.getResponseHeader("Content-Type");
 170 
 
 171 
                if (h != null) {
 172 
                        HeaderElement[] elements = h.getElements();
 173 
 
 174 
                        for (HeaderElement element : elements) {
 175 
                                String name = element.getName().toUpperCase();
 176 
                                if (name.startsWith("MULTIPART/")) {
 177 
                                        NameValuePair parameter = element
 178 
                                                        .getParameterByName("boundary");
 179 
                                        if (parameter != null) {
 180 
                                                boundary = parameter.getValue();
 181 
                                        }
 182 
                                }
 183 
                        }
 184 
                }
 185 
 
 186 
                return boundary;
 187 
        }
 188 
       
 189 
        public static long getContentLength(HttpMethod httpMethod) {
 190 
                Header h = null;
 191 
                String contentLen = "0";
 192 
 
 193 
                h = httpMethod.getResponseHeader("Content-Length");
 194 
                if (h != null) {
 195 
                        contentLen = h.getValue();
 196 
                }
 197 
               
 198 
                long contentLength = Long.valueOf(contentLen).longValue();
 199 
               
 200 
                return contentLength;
 201 
        }
 202 
 
 203 
        public static void saveAttachment(BufferedInputStream responseStream,
 204 
                String boundary, long totalBytes, long totalBytesRead) {
 205 
                try {
 206 
                        if (responseStream != null) {
 207 
                                BufferedOutputStream out = new BufferedOutputStream(
 208 
                                                new FileOutputStream(
 209 
                                                                "c:\\temp\\test.txt"));
 210 
                                byte data[] = new byte[4096];
 211 
                                int bytesread = -1;
 212 
                                boolean boundaryEndHit = false;
 213 
                                int boundaryEnd = boundary.length()+6;
 214 
                               
 215 
                                System.out.println("Attachment Starts Here ========= ");
 216 
                               
 217 
                                while ((bytesread = responseStream.read(data)) != -1) {
 218 
                                        totalBytesRead += bytesread;
 219 
                                       
 220 
                                        if (totalBytesRead <= (totalBytes -
 221 
                                                boundaryEnd)) {
 222 
                                                out.write(data, 0, bytesread);
 223 
                                        } else if (totalBytesRead > (totalBytes -
 224 
                                                boundaryEnd)) {
 225 
                                                out.write(data, 0, bytesread -
 226 
                                                        (int) (totalBytesRead -
 227 
                                                                (totalBytes - boundaryEnd)));
 228 
                                        }
 229 
                                }
 230 
                               
 231 
                                responseStream.close();
 232 
                                out.close();
 233 
                        }
 234 
                } catch (Exception e) {
 235 
                        e.printStackTrace();
 236 
                }
 237 
        }
 238 
}

The example was tested with 10 concurrent thread each downloaded 160MB of file from web service and the JVM memory utilization was mere 20MB.