httpresponse - Receive partial file(sometimes) when reading from Google Storage using HTTP Response -


i trying read files google storage , write files in our filesystem (hdfs). if run period of time (lets 7 days), full file lines matching whats on source , partial files (discrepancy quite large). pasting below method takes response , writes file. or suggestions how can troubleshoot further appreciated.

thanks,

before calling method simple check on response status code -

if(response.getstatuscode() == 200 &&                     stringutils.equals(response.getcontenttype(), "application/zip")) {                     writehdfsfile(response, path); }  private void writehdfsfile(httpresponse response, string path) throws ioexception {         final gzipinputstream inputstream = new gzipinputstream(response.getcontent());         path filepath = new path(path);         final fsdataoutputstream outputstream = filesystem.create(filepath, true);         final byte[] buffer = new byte[1024];         int length;         try {             while((length = inputstream.read(buffer)) > 0) {                 outputstream.write(buffer, 0, length);             }             outputstream.flush();         } {             inputstream.close();             outputstream.close();         }     } 

the way solved downloading file first , unzipping , writing it. basically, splitting 2 steps solved issue. if else ran same issue..


Comments

Popular posts from this blog

android - getbluetoothservice() called with no bluetoothmanagercallback -

sql - ASP.NET SqlDataSource, like on SelectCommand -

ios - Undefined symbols for architecture armv7: "_OBJC_CLASS_$_SSZipArchive" -