How to ZIP (compress) an SQLITE database file in Cordova?












5















I need a "support" mode for my Cordova app currently running on Windows Mobile and iOS. For this purpose, I need to compress an sqlite database file and upload it to a server. The database has to be compressed as it might grow over 250MB and the upload has to work without a wifi connection.



Searching the web brought up different approaches but all of them were outdated or did only solve my problem for either iOS or Windows Mobile. For example, when using the Cordova file plug-in I've encountered this in the plug-in documentation:




Supported Platforms



Android iOS OS X Windows* Browser




  • These platforms do not support FileReader.readAsArrayBuffer nor FileWriter.write(blob).




This was my approach: Cordova - Zip files and folders on iOS



Any ideas?










share|improve this question





























    5















    I need a "support" mode for my Cordova app currently running on Windows Mobile and iOS. For this purpose, I need to compress an sqlite database file and upload it to a server. The database has to be compressed as it might grow over 250MB and the upload has to work without a wifi connection.



    Searching the web brought up different approaches but all of them were outdated or did only solve my problem for either iOS or Windows Mobile. For example, when using the Cordova file plug-in I've encountered this in the plug-in documentation:




    Supported Platforms



    Android iOS OS X Windows* Browser




    • These platforms do not support FileReader.readAsArrayBuffer nor FileWriter.write(blob).




    This was my approach: Cordova - Zip files and folders on iOS



    Any ideas?










    share|improve this question



























      5












      5








      5


      1






      I need a "support" mode for my Cordova app currently running on Windows Mobile and iOS. For this purpose, I need to compress an sqlite database file and upload it to a server. The database has to be compressed as it might grow over 250MB and the upload has to work without a wifi connection.



      Searching the web brought up different approaches but all of them were outdated or did only solve my problem for either iOS or Windows Mobile. For example, when using the Cordova file plug-in I've encountered this in the plug-in documentation:




      Supported Platforms



      Android iOS OS X Windows* Browser




      • These platforms do not support FileReader.readAsArrayBuffer nor FileWriter.write(blob).




      This was my approach: Cordova - Zip files and folders on iOS



      Any ideas?










      share|improve this question
















      I need a "support" mode for my Cordova app currently running on Windows Mobile and iOS. For this purpose, I need to compress an sqlite database file and upload it to a server. The database has to be compressed as it might grow over 250MB and the upload has to work without a wifi connection.



      Searching the web brought up different approaches but all of them were outdated or did only solve my problem for either iOS or Windows Mobile. For example, when using the Cordova file plug-in I've encountered this in the plug-in documentation:




      Supported Platforms



      Android iOS OS X Windows* Browser




      • These platforms do not support FileReader.readAsArrayBuffer nor FileWriter.write(blob).




      This was my approach: Cordova - Zip files and folders on iOS



      Any ideas?







      ios sqlite cordova zip windows-mobile






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 6 '18 at 13:04







      MichaelS

















      asked Nov 2 '18 at 9:28









      MichaelSMichaelS

      4,83752438




      4,83752438
























          2 Answers
          2






          active

          oldest

          votes


















          2














          I would suggest you to give FileReader() a second chance.



          In my case wich may be very similar to yours, I read a file using FilerReader.readAsArrayBuffer and after that, compress it using the JSZip library: http://stuartk.com/jszip



          In opposition to the cordova-file-plugin's API documentation (https://cordova.apache.org/docs/en/latest/reference/cordova-plugin-file/)
          "Windows*"->"These platforms do not support FileReader.readAsArrayBuffer nor FileWriter.write(blob))"
          I experienced that readAsArrayBuffer works under Windows UWP platform, but slower.



          So in my case, with a file of approx. 50M I had to wait for nearly 2min for the whole process to finish!



          Try following this example:



          You'll need to adapt to your paths but this runs for WINDOWS UWP and IOS (didn't test it with Android but that was not your question).



          Also, you'll need to implement your own error handler (errorHandler).
          This solution uses Promises since you'll have to wait for the file beeing read and compressed.



          PS1: Always be sure your "device ready's event" as been fired in order to access plugins.



          PS2: You may encouter no access permission on the database file, this may be related to the fact, that it's being used by another process.
          Be sure the database is closed.
          SQLITE:



                  var sqlite = window.sqlitePlugin.openDatabase({ name: 'yourdb.db', location: 'default' });
          sqlite.close(function () {
          console.log("DONE closing db");
          }, function (error) {
          console.log("ERROR closing db");
          console.log(JSON.stringify(error));
          });


          "ZIP" function:



              function zipFile(sourceFileName, targetFileName) {
          return new Promise(function (resolve, reject) {
          window.requestFileSystem(LocalFileSystem.PERSISTENT, 0, function (fs) {
          fs.root.getFile(sourceFileName, { create: false, exclusive: false }, function (fe) {
          fe.file(function (file) {
          var reader = new FileReader();
          reader.onloadend = function (data) {
          var zip = new JSZip();
          zip.file(sourceFileName, data.target.result);
          zip.generateAsync({
          type: "blob",
          compression: "DEFLATE",
          compressionOptions: {
          level: 9
          }
          // level 9 means max. compression
          // this may also take some time depending on the size of your file
          // I tested it with a 50M file, it took about 65 sec.
          }).then(

          // following is post-zip in order to transfer the file to a server
          function (blob) {
          fs.root.getFile(targetFileName, { create: true, exclusive: false }, function (newzip) {
          writeFile(newzip, blob, "application/zip").then(function () {
          var f = blob;
          var zipReader = new FileReader();

          zipReader.onloadend = function (theFile) {
          var base64 = window.btoa(theFile.target.result);
          resolve(base64);
          };
          // need to "resolve" the zipped file as base64 in order to incluse it in my REST post (server-upload)
          zipReader.readAsBinaryString(f);
          });

          });

          }
          )
          };
          reader.readAsArrayBuffer(file);
          // this may take some time depending on the size of your file
          // I tested it with a 50M file, it took about 72 sec.
          }, errorHandler);
          }, errorHandler);
          });
          });
          }


          ex call:



          if (window.cordova) {
          document.addEventListener('deviceready', function () {
          zipFile("yourDatabaseFileName.db","compressedDatabaseFile.zip");
          });
          }





          share|improve this answer































            1














            Why not use sqlite ".dump" command as query and get result via steam and then compress the output. Even though text dump will be larger it will get reasonable size when you compress it. I think there are some very good text only compression algorithms as well,






            share|improve this answer
























              Your Answer






              StackExchange.ifUsing("editor", function () {
              StackExchange.using("externalEditor", function () {
              StackExchange.using("snippets", function () {
              StackExchange.snippets.init();
              });
              });
              }, "code-snippets");

              StackExchange.ready(function() {
              var channelOptions = {
              tags: "".split(" "),
              id: "1"
              };
              initTagRenderer("".split(" "), "".split(" "), channelOptions);

              StackExchange.using("externalEditor", function() {
              // Have to fire editor after snippets, if snippets enabled
              if (StackExchange.settings.snippets.snippetsEnabled) {
              StackExchange.using("snippets", function() {
              createEditor();
              });
              }
              else {
              createEditor();
              }
              });

              function createEditor() {
              StackExchange.prepareEditor({
              heartbeatType: 'answer',
              autoActivateHeartbeat: false,
              convertImagesToLinks: true,
              noModals: true,
              showLowRepImageUploadWarning: true,
              reputationToPostImages: 10,
              bindNavPrevention: true,
              postfix: "",
              imageUploader: {
              brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
              contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
              allowUrls: true
              },
              onDemand: true,
              discardSelector: ".discard-answer"
              ,immediatelyShowMarkdownHelp:true
              });


              }
              });














              draft saved

              draft discarded


















              StackExchange.ready(
              function () {
              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53115869%2fhow-to-zip-compress-an-sqlite-database-file-in-cordova%23new-answer', 'question_page');
              }
              );

              Post as a guest















              Required, but never shown

























              2 Answers
              2






              active

              oldest

              votes








              2 Answers
              2






              active

              oldest

              votes









              active

              oldest

              votes






              active

              oldest

              votes









              2














              I would suggest you to give FileReader() a second chance.



              In my case wich may be very similar to yours, I read a file using FilerReader.readAsArrayBuffer and after that, compress it using the JSZip library: http://stuartk.com/jszip



              In opposition to the cordova-file-plugin's API documentation (https://cordova.apache.org/docs/en/latest/reference/cordova-plugin-file/)
              "Windows*"->"These platforms do not support FileReader.readAsArrayBuffer nor FileWriter.write(blob))"
              I experienced that readAsArrayBuffer works under Windows UWP platform, but slower.



              So in my case, with a file of approx. 50M I had to wait for nearly 2min for the whole process to finish!



              Try following this example:



              You'll need to adapt to your paths but this runs for WINDOWS UWP and IOS (didn't test it with Android but that was not your question).



              Also, you'll need to implement your own error handler (errorHandler).
              This solution uses Promises since you'll have to wait for the file beeing read and compressed.



              PS1: Always be sure your "device ready's event" as been fired in order to access plugins.



              PS2: You may encouter no access permission on the database file, this may be related to the fact, that it's being used by another process.
              Be sure the database is closed.
              SQLITE:



                      var sqlite = window.sqlitePlugin.openDatabase({ name: 'yourdb.db', location: 'default' });
              sqlite.close(function () {
              console.log("DONE closing db");
              }, function (error) {
              console.log("ERROR closing db");
              console.log(JSON.stringify(error));
              });


              "ZIP" function:



                  function zipFile(sourceFileName, targetFileName) {
              return new Promise(function (resolve, reject) {
              window.requestFileSystem(LocalFileSystem.PERSISTENT, 0, function (fs) {
              fs.root.getFile(sourceFileName, { create: false, exclusive: false }, function (fe) {
              fe.file(function (file) {
              var reader = new FileReader();
              reader.onloadend = function (data) {
              var zip = new JSZip();
              zip.file(sourceFileName, data.target.result);
              zip.generateAsync({
              type: "blob",
              compression: "DEFLATE",
              compressionOptions: {
              level: 9
              }
              // level 9 means max. compression
              // this may also take some time depending on the size of your file
              // I tested it with a 50M file, it took about 65 sec.
              }).then(

              // following is post-zip in order to transfer the file to a server
              function (blob) {
              fs.root.getFile(targetFileName, { create: true, exclusive: false }, function (newzip) {
              writeFile(newzip, blob, "application/zip").then(function () {
              var f = blob;
              var zipReader = new FileReader();

              zipReader.onloadend = function (theFile) {
              var base64 = window.btoa(theFile.target.result);
              resolve(base64);
              };
              // need to "resolve" the zipped file as base64 in order to incluse it in my REST post (server-upload)
              zipReader.readAsBinaryString(f);
              });

              });

              }
              )
              };
              reader.readAsArrayBuffer(file);
              // this may take some time depending on the size of your file
              // I tested it with a 50M file, it took about 72 sec.
              }, errorHandler);
              }, errorHandler);
              });
              });
              }


              ex call:



              if (window.cordova) {
              document.addEventListener('deviceready', function () {
              zipFile("yourDatabaseFileName.db","compressedDatabaseFile.zip");
              });
              }





              share|improve this answer




























                2














                I would suggest you to give FileReader() a second chance.



                In my case wich may be very similar to yours, I read a file using FilerReader.readAsArrayBuffer and after that, compress it using the JSZip library: http://stuartk.com/jszip



                In opposition to the cordova-file-plugin's API documentation (https://cordova.apache.org/docs/en/latest/reference/cordova-plugin-file/)
                "Windows*"->"These platforms do not support FileReader.readAsArrayBuffer nor FileWriter.write(blob))"
                I experienced that readAsArrayBuffer works under Windows UWP platform, but slower.



                So in my case, with a file of approx. 50M I had to wait for nearly 2min for the whole process to finish!



                Try following this example:



                You'll need to adapt to your paths but this runs for WINDOWS UWP and IOS (didn't test it with Android but that was not your question).



                Also, you'll need to implement your own error handler (errorHandler).
                This solution uses Promises since you'll have to wait for the file beeing read and compressed.



                PS1: Always be sure your "device ready's event" as been fired in order to access plugins.



                PS2: You may encouter no access permission on the database file, this may be related to the fact, that it's being used by another process.
                Be sure the database is closed.
                SQLITE:



                        var sqlite = window.sqlitePlugin.openDatabase({ name: 'yourdb.db', location: 'default' });
                sqlite.close(function () {
                console.log("DONE closing db");
                }, function (error) {
                console.log("ERROR closing db");
                console.log(JSON.stringify(error));
                });


                "ZIP" function:



                    function zipFile(sourceFileName, targetFileName) {
                return new Promise(function (resolve, reject) {
                window.requestFileSystem(LocalFileSystem.PERSISTENT, 0, function (fs) {
                fs.root.getFile(sourceFileName, { create: false, exclusive: false }, function (fe) {
                fe.file(function (file) {
                var reader = new FileReader();
                reader.onloadend = function (data) {
                var zip = new JSZip();
                zip.file(sourceFileName, data.target.result);
                zip.generateAsync({
                type: "blob",
                compression: "DEFLATE",
                compressionOptions: {
                level: 9
                }
                // level 9 means max. compression
                // this may also take some time depending on the size of your file
                // I tested it with a 50M file, it took about 65 sec.
                }).then(

                // following is post-zip in order to transfer the file to a server
                function (blob) {
                fs.root.getFile(targetFileName, { create: true, exclusive: false }, function (newzip) {
                writeFile(newzip, blob, "application/zip").then(function () {
                var f = blob;
                var zipReader = new FileReader();

                zipReader.onloadend = function (theFile) {
                var base64 = window.btoa(theFile.target.result);
                resolve(base64);
                };
                // need to "resolve" the zipped file as base64 in order to incluse it in my REST post (server-upload)
                zipReader.readAsBinaryString(f);
                });

                });

                }
                )
                };
                reader.readAsArrayBuffer(file);
                // this may take some time depending on the size of your file
                // I tested it with a 50M file, it took about 72 sec.
                }, errorHandler);
                }, errorHandler);
                });
                });
                }


                ex call:



                if (window.cordova) {
                document.addEventListener('deviceready', function () {
                zipFile("yourDatabaseFileName.db","compressedDatabaseFile.zip");
                });
                }





                share|improve this answer


























                  2












                  2








                  2







                  I would suggest you to give FileReader() a second chance.



                  In my case wich may be very similar to yours, I read a file using FilerReader.readAsArrayBuffer and after that, compress it using the JSZip library: http://stuartk.com/jszip



                  In opposition to the cordova-file-plugin's API documentation (https://cordova.apache.org/docs/en/latest/reference/cordova-plugin-file/)
                  "Windows*"->"These platforms do not support FileReader.readAsArrayBuffer nor FileWriter.write(blob))"
                  I experienced that readAsArrayBuffer works under Windows UWP platform, but slower.



                  So in my case, with a file of approx. 50M I had to wait for nearly 2min for the whole process to finish!



                  Try following this example:



                  You'll need to adapt to your paths but this runs for WINDOWS UWP and IOS (didn't test it with Android but that was not your question).



                  Also, you'll need to implement your own error handler (errorHandler).
                  This solution uses Promises since you'll have to wait for the file beeing read and compressed.



                  PS1: Always be sure your "device ready's event" as been fired in order to access plugins.



                  PS2: You may encouter no access permission on the database file, this may be related to the fact, that it's being used by another process.
                  Be sure the database is closed.
                  SQLITE:



                          var sqlite = window.sqlitePlugin.openDatabase({ name: 'yourdb.db', location: 'default' });
                  sqlite.close(function () {
                  console.log("DONE closing db");
                  }, function (error) {
                  console.log("ERROR closing db");
                  console.log(JSON.stringify(error));
                  });


                  "ZIP" function:



                      function zipFile(sourceFileName, targetFileName) {
                  return new Promise(function (resolve, reject) {
                  window.requestFileSystem(LocalFileSystem.PERSISTENT, 0, function (fs) {
                  fs.root.getFile(sourceFileName, { create: false, exclusive: false }, function (fe) {
                  fe.file(function (file) {
                  var reader = new FileReader();
                  reader.onloadend = function (data) {
                  var zip = new JSZip();
                  zip.file(sourceFileName, data.target.result);
                  zip.generateAsync({
                  type: "blob",
                  compression: "DEFLATE",
                  compressionOptions: {
                  level: 9
                  }
                  // level 9 means max. compression
                  // this may also take some time depending on the size of your file
                  // I tested it with a 50M file, it took about 65 sec.
                  }).then(

                  // following is post-zip in order to transfer the file to a server
                  function (blob) {
                  fs.root.getFile(targetFileName, { create: true, exclusive: false }, function (newzip) {
                  writeFile(newzip, blob, "application/zip").then(function () {
                  var f = blob;
                  var zipReader = new FileReader();

                  zipReader.onloadend = function (theFile) {
                  var base64 = window.btoa(theFile.target.result);
                  resolve(base64);
                  };
                  // need to "resolve" the zipped file as base64 in order to incluse it in my REST post (server-upload)
                  zipReader.readAsBinaryString(f);
                  });

                  });

                  }
                  )
                  };
                  reader.readAsArrayBuffer(file);
                  // this may take some time depending on the size of your file
                  // I tested it with a 50M file, it took about 72 sec.
                  }, errorHandler);
                  }, errorHandler);
                  });
                  });
                  }


                  ex call:



                  if (window.cordova) {
                  document.addEventListener('deviceready', function () {
                  zipFile("yourDatabaseFileName.db","compressedDatabaseFile.zip");
                  });
                  }





                  share|improve this answer













                  I would suggest you to give FileReader() a second chance.



                  In my case wich may be very similar to yours, I read a file using FilerReader.readAsArrayBuffer and after that, compress it using the JSZip library: http://stuartk.com/jszip



                  In opposition to the cordova-file-plugin's API documentation (https://cordova.apache.org/docs/en/latest/reference/cordova-plugin-file/)
                  "Windows*"->"These platforms do not support FileReader.readAsArrayBuffer nor FileWriter.write(blob))"
                  I experienced that readAsArrayBuffer works under Windows UWP platform, but slower.



                  So in my case, with a file of approx. 50M I had to wait for nearly 2min for the whole process to finish!



                  Try following this example:



                  You'll need to adapt to your paths but this runs for WINDOWS UWP and IOS (didn't test it with Android but that was not your question).



                  Also, you'll need to implement your own error handler (errorHandler).
                  This solution uses Promises since you'll have to wait for the file beeing read and compressed.



                  PS1: Always be sure your "device ready's event" as been fired in order to access plugins.



                  PS2: You may encouter no access permission on the database file, this may be related to the fact, that it's being used by another process.
                  Be sure the database is closed.
                  SQLITE:



                          var sqlite = window.sqlitePlugin.openDatabase({ name: 'yourdb.db', location: 'default' });
                  sqlite.close(function () {
                  console.log("DONE closing db");
                  }, function (error) {
                  console.log("ERROR closing db");
                  console.log(JSON.stringify(error));
                  });


                  "ZIP" function:



                      function zipFile(sourceFileName, targetFileName) {
                  return new Promise(function (resolve, reject) {
                  window.requestFileSystem(LocalFileSystem.PERSISTENT, 0, function (fs) {
                  fs.root.getFile(sourceFileName, { create: false, exclusive: false }, function (fe) {
                  fe.file(function (file) {
                  var reader = new FileReader();
                  reader.onloadend = function (data) {
                  var zip = new JSZip();
                  zip.file(sourceFileName, data.target.result);
                  zip.generateAsync({
                  type: "blob",
                  compression: "DEFLATE",
                  compressionOptions: {
                  level: 9
                  }
                  // level 9 means max. compression
                  // this may also take some time depending on the size of your file
                  // I tested it with a 50M file, it took about 65 sec.
                  }).then(

                  // following is post-zip in order to transfer the file to a server
                  function (blob) {
                  fs.root.getFile(targetFileName, { create: true, exclusive: false }, function (newzip) {
                  writeFile(newzip, blob, "application/zip").then(function () {
                  var f = blob;
                  var zipReader = new FileReader();

                  zipReader.onloadend = function (theFile) {
                  var base64 = window.btoa(theFile.target.result);
                  resolve(base64);
                  };
                  // need to "resolve" the zipped file as base64 in order to incluse it in my REST post (server-upload)
                  zipReader.readAsBinaryString(f);
                  });

                  });

                  }
                  )
                  };
                  reader.readAsArrayBuffer(file);
                  // this may take some time depending on the size of your file
                  // I tested it with a 50M file, it took about 72 sec.
                  }, errorHandler);
                  }, errorHandler);
                  });
                  });
                  }


                  ex call:



                  if (window.cordova) {
                  document.addEventListener('deviceready', function () {
                  zipFile("yourDatabaseFileName.db","compressedDatabaseFile.zip");
                  });
                  }






                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Nov 16 '18 at 11:28









                  drallebasdrallebas

                  362




                  362

























                      1














                      Why not use sqlite ".dump" command as query and get result via steam and then compress the output. Even though text dump will be larger it will get reasonable size when you compress it. I think there are some very good text only compression algorithms as well,






                      share|improve this answer




























                        1














                        Why not use sqlite ".dump" command as query and get result via steam and then compress the output. Even though text dump will be larger it will get reasonable size when you compress it. I think there are some very good text only compression algorithms as well,






                        share|improve this answer


























                          1












                          1








                          1







                          Why not use sqlite ".dump" command as query and get result via steam and then compress the output. Even though text dump will be larger it will get reasonable size when you compress it. I think there are some very good text only compression algorithms as well,






                          share|improve this answer













                          Why not use sqlite ".dump" command as query and get result via steam and then compress the output. Even though text dump will be larger it will get reasonable size when you compress it. I think there are some very good text only compression algorithms as well,







                          share|improve this answer












                          share|improve this answer



                          share|improve this answer










                          answered Nov 11 '18 at 14:35









                          AbdurrahimAbdurrahim

                          1,0791012




                          1,0791012






























                              draft saved

                              draft discarded




















































                              Thanks for contributing an answer to Stack Overflow!


                              • Please be sure to answer the question. Provide details and share your research!

                              But avoid



                              • Asking for help, clarification, or responding to other answers.

                              • Making statements based on opinion; back them up with references or personal experience.


                              To learn more, see our tips on writing great answers.




                              draft saved


                              draft discarded














                              StackExchange.ready(
                              function () {
                              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53115869%2fhow-to-zip-compress-an-sqlite-database-file-in-cordova%23new-answer', 'question_page');
                              }
                              );

                              Post as a guest















                              Required, but never shown





















































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown

































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown







                              Popular posts from this blog

                              Xamarin.iOS Cant Deploy on Iphone

                              Glorious Revolution

                              Dulmage-Mendelsohn matrix decomposition in Python