Most Appropriate Database for Distributed Platform with (regular) Temporarily Unreachable Primary (cloud)...












0















I'm currently working on a project that will have two database levels to in its topology:




  1. Global Primary Use Database - This primary database will be for a parent organization. This database will store users, set up events, web page data, and various other high-level information that will be used to populate public facing web pages. They will house informational data as well as be a destination for event information and a place people can register for those events.


  2. Regional Use Databases - This secondary database will pull data from the primary database (event information, primarily) and use that data on a local server that is running at the event. That server will be used to manage registration information (payment, user data, etc) as well as collect data real-time at the event (these events will be competitive events where we'll be capturing timing and scoring data).



In most cases, we should have a live connection between the local site and the primary database servers and could likely deal with most of the registration data directly on that server. However, in some cases where connectivity is poor or non-existent, it's possible we'll need to pull that data from the primary server before the event on-site check-in starts and work from a local database. That local database will then be connected to a local wifi network that can distribute results to participants at the event (event size varies between 50 and 1400 people - no more than 40 people are being timed and scored at one). After the event, we would then need to connect back to the primary database servers to sync with that data (pushing data up to the primary servers). The system should be able to be set up where the data that's pulled from the primary server goes unchanged until the time when the re-sync happens so we shouldn't have to worry about pulling any data.



So... what database am I using?



After doing some searching, it seems like CouchDB is particularly good at reconciling multilevel systems like this and coming up with logic to sync the databases from when connectivity stops to a restart. However, as it's possible we'll be updating data fairly frequently because of how our timing and scoring works, I like some DBs ability to work with partial document writes (Patching, for example in MongoDB). This seems like it's a bit more of a data modeling issue to make sure data is set up so multiple users can write to the appropriate data without breaking the data (for example, generally two users work to monitor the live data - one entering in participant data and another monitoring those participants for penalties).



I would love to use something I can stick a GraphQL server in front of, but it's not an absolute requirement at the moment. The front end is all written in React and the backend is all Node.js. I've used Google Cloud for my test platform, but it may move to AWS (I'm not dedicated to a given cloud platform yet to run this all). Most of my proof of concept stuff is all being housed via a Loopback server to help me model everything. I've been looking at Neo4j on the graph side of things too and AWS AppSync (I love how it feels like "state on a server"), but it's that "local" layer that's giving me some pause as to a final choice.



Thanks!










share|improve this question



























    0















    I'm currently working on a project that will have two database levels to in its topology:




    1. Global Primary Use Database - This primary database will be for a parent organization. This database will store users, set up events, web page data, and various other high-level information that will be used to populate public facing web pages. They will house informational data as well as be a destination for event information and a place people can register for those events.


    2. Regional Use Databases - This secondary database will pull data from the primary database (event information, primarily) and use that data on a local server that is running at the event. That server will be used to manage registration information (payment, user data, etc) as well as collect data real-time at the event (these events will be competitive events where we'll be capturing timing and scoring data).



    In most cases, we should have a live connection between the local site and the primary database servers and could likely deal with most of the registration data directly on that server. However, in some cases where connectivity is poor or non-existent, it's possible we'll need to pull that data from the primary server before the event on-site check-in starts and work from a local database. That local database will then be connected to a local wifi network that can distribute results to participants at the event (event size varies between 50 and 1400 people - no more than 40 people are being timed and scored at one). After the event, we would then need to connect back to the primary database servers to sync with that data (pushing data up to the primary servers). The system should be able to be set up where the data that's pulled from the primary server goes unchanged until the time when the re-sync happens so we shouldn't have to worry about pulling any data.



    So... what database am I using?



    After doing some searching, it seems like CouchDB is particularly good at reconciling multilevel systems like this and coming up with logic to sync the databases from when connectivity stops to a restart. However, as it's possible we'll be updating data fairly frequently because of how our timing and scoring works, I like some DBs ability to work with partial document writes (Patching, for example in MongoDB). This seems like it's a bit more of a data modeling issue to make sure data is set up so multiple users can write to the appropriate data without breaking the data (for example, generally two users work to monitor the live data - one entering in participant data and another monitoring those participants for penalties).



    I would love to use something I can stick a GraphQL server in front of, but it's not an absolute requirement at the moment. The front end is all written in React and the backend is all Node.js. I've used Google Cloud for my test platform, but it may move to AWS (I'm not dedicated to a given cloud platform yet to run this all). Most of my proof of concept stuff is all being housed via a Loopback server to help me model everything. I've been looking at Neo4j on the graph side of things too and AWS AppSync (I love how it feels like "state on a server"), but it's that "local" layer that's giving me some pause as to a final choice.



    Thanks!










    share|improve this question

























      0












      0








      0








      I'm currently working on a project that will have two database levels to in its topology:




      1. Global Primary Use Database - This primary database will be for a parent organization. This database will store users, set up events, web page data, and various other high-level information that will be used to populate public facing web pages. They will house informational data as well as be a destination for event information and a place people can register for those events.


      2. Regional Use Databases - This secondary database will pull data from the primary database (event information, primarily) and use that data on a local server that is running at the event. That server will be used to manage registration information (payment, user data, etc) as well as collect data real-time at the event (these events will be competitive events where we'll be capturing timing and scoring data).



      In most cases, we should have a live connection between the local site and the primary database servers and could likely deal with most of the registration data directly on that server. However, in some cases where connectivity is poor or non-existent, it's possible we'll need to pull that data from the primary server before the event on-site check-in starts and work from a local database. That local database will then be connected to a local wifi network that can distribute results to participants at the event (event size varies between 50 and 1400 people - no more than 40 people are being timed and scored at one). After the event, we would then need to connect back to the primary database servers to sync with that data (pushing data up to the primary servers). The system should be able to be set up where the data that's pulled from the primary server goes unchanged until the time when the re-sync happens so we shouldn't have to worry about pulling any data.



      So... what database am I using?



      After doing some searching, it seems like CouchDB is particularly good at reconciling multilevel systems like this and coming up with logic to sync the databases from when connectivity stops to a restart. However, as it's possible we'll be updating data fairly frequently because of how our timing and scoring works, I like some DBs ability to work with partial document writes (Patching, for example in MongoDB). This seems like it's a bit more of a data modeling issue to make sure data is set up so multiple users can write to the appropriate data without breaking the data (for example, generally two users work to monitor the live data - one entering in participant data and another monitoring those participants for penalties).



      I would love to use something I can stick a GraphQL server in front of, but it's not an absolute requirement at the moment. The front end is all written in React and the backend is all Node.js. I've used Google Cloud for my test platform, but it may move to AWS (I'm not dedicated to a given cloud platform yet to run this all). Most of my proof of concept stuff is all being housed via a Loopback server to help me model everything. I've been looking at Neo4j on the graph side of things too and AWS AppSync (I love how it feels like "state on a server"), but it's that "local" layer that's giving me some pause as to a final choice.



      Thanks!










      share|improve this question














      I'm currently working on a project that will have two database levels to in its topology:




      1. Global Primary Use Database - This primary database will be for a parent organization. This database will store users, set up events, web page data, and various other high-level information that will be used to populate public facing web pages. They will house informational data as well as be a destination for event information and a place people can register for those events.


      2. Regional Use Databases - This secondary database will pull data from the primary database (event information, primarily) and use that data on a local server that is running at the event. That server will be used to manage registration information (payment, user data, etc) as well as collect data real-time at the event (these events will be competitive events where we'll be capturing timing and scoring data).



      In most cases, we should have a live connection between the local site and the primary database servers and could likely deal with most of the registration data directly on that server. However, in some cases where connectivity is poor or non-existent, it's possible we'll need to pull that data from the primary server before the event on-site check-in starts and work from a local database. That local database will then be connected to a local wifi network that can distribute results to participants at the event (event size varies between 50 and 1400 people - no more than 40 people are being timed and scored at one). After the event, we would then need to connect back to the primary database servers to sync with that data (pushing data up to the primary servers). The system should be able to be set up where the data that's pulled from the primary server goes unchanged until the time when the re-sync happens so we shouldn't have to worry about pulling any data.



      So... what database am I using?



      After doing some searching, it seems like CouchDB is particularly good at reconciling multilevel systems like this and coming up with logic to sync the databases from when connectivity stops to a restart. However, as it's possible we'll be updating data fairly frequently because of how our timing and scoring works, I like some DBs ability to work with partial document writes (Patching, for example in MongoDB). This seems like it's a bit more of a data modeling issue to make sure data is set up so multiple users can write to the appropriate data without breaking the data (for example, generally two users work to monitor the live data - one entering in participant data and another monitoring those participants for penalties).



      I would love to use something I can stick a GraphQL server in front of, but it's not an absolute requirement at the moment. The front end is all written in React and the backend is all Node.js. I've used Google Cloud for my test platform, but it may move to AWS (I'm not dedicated to a given cloud platform yet to run this all). Most of my proof of concept stuff is all being housed via a Loopback server to help me model everything. I've been looking at Neo4j on the graph side of things too and AWS AppSync (I love how it feels like "state on a server"), but it's that "local" layer that's giving me some pause as to a final choice.



      Thanks!







      reactjs mongodb couchdb graphql loopback






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 16 '18 at 4:51









      Randall PrinceRandall Prince

      61




      61
























          0






          active

          oldest

          votes











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53331643%2fmost-appropriate-database-for-distributed-platform-with-regular-temporarily-un%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53331643%2fmost-appropriate-database-for-distributed-platform-with-regular-temporarily-un%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Xamarin.iOS Cant Deploy on Iphone

          Glorious Revolution

          Dulmage-Mendelsohn matrix decomposition in Python