ML.net Add New Data to Exist Generated Model












2















i generated a ML Model Using ML.Net V 0.7.0 last version



i need to add a new learning data to this existing Model without regenerated it with the new and old data



as i have a large data set that exceed 100 Million Record



and i need to add 100 record without reload all last data set to generate the new model



any ideas please



this is critical for me



best regards










share|improve this question



























    2















    i generated a ML Model Using ML.Net V 0.7.0 last version



    i need to add a new learning data to this existing Model without regenerated it with the new and old data



    as i have a large data set that exceed 100 Million Record



    and i need to add 100 record without reload all last data set to generate the new model



    any ideas please



    this is critical for me



    best regards










    share|improve this question

























      2












      2








      2


      1






      i generated a ML Model Using ML.Net V 0.7.0 last version



      i need to add a new learning data to this existing Model without regenerated it with the new and old data



      as i have a large data set that exceed 100 Million Record



      and i need to add 100 record without reload all last data set to generate the new model



      any ideas please



      this is critical for me



      best regards










      share|improve this question














      i generated a ML Model Using ML.Net V 0.7.0 last version



      i need to add a new learning data to this existing Model without regenerated it with the new and old data



      as i have a large data set that exceed 100 Million Record



      and i need to add 100 record without reload all last data set to generate the new model



      any ideas please



      this is critical for me



      best regards







      ml.net






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 15 '18 at 11:50









      NobDevNobDev

      113




      113
























          1 Answer
          1






          active

          oldest

          votes


















          4














          Some trainers in ML.NET support training with an initial predictor, which means you can use an existing predictor as the starting point for training with new data.



          A test showing this can be found here with the relevant code being:



          // Train the first predictor.
          var trainer = ml.BinaryClassification.Trainers.StochasticDualCoordinateAscent("Label", "Features",advancedSettings: s => s.NumThreads = 1);
          var firstModel = trainer.Fit(trainData);

          // Train the second predictor on the same data.
          var secondTrainer = ml.BinaryClassification.Trainers.AveragedPerceptron("Label","Features");

          var trainRoles = new RoleMappedData(trainData, label: "Label", feature: "Features");
          var finalModel = secondTrainer.Train(new TrainContext(trainRoles, initialPredictor: firstModel.Model));





          share|improve this answer
























          • I think the OP is interested in adding more data to the first predictor, not creating a second type of predictor. I'm interested in this as well.

            – Webjedi
            Jan 4 at 15:55











          • Is it as simple as train model1 and model2 and then do finalmodel=model1.append(model2)? If both models are trained the same way?

            – Webjedi
            Jan 4 at 17:27











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53318863%2fml-net-add-new-data-to-exist-generated-model%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          4














          Some trainers in ML.NET support training with an initial predictor, which means you can use an existing predictor as the starting point for training with new data.



          A test showing this can be found here with the relevant code being:



          // Train the first predictor.
          var trainer = ml.BinaryClassification.Trainers.StochasticDualCoordinateAscent("Label", "Features",advancedSettings: s => s.NumThreads = 1);
          var firstModel = trainer.Fit(trainData);

          // Train the second predictor on the same data.
          var secondTrainer = ml.BinaryClassification.Trainers.AveragedPerceptron("Label","Features");

          var trainRoles = new RoleMappedData(trainData, label: "Label", feature: "Features");
          var finalModel = secondTrainer.Train(new TrainContext(trainRoles, initialPredictor: firstModel.Model));





          share|improve this answer
























          • I think the OP is interested in adding more data to the first predictor, not creating a second type of predictor. I'm interested in this as well.

            – Webjedi
            Jan 4 at 15:55











          • Is it as simple as train model1 and model2 and then do finalmodel=model1.append(model2)? If both models are trained the same way?

            – Webjedi
            Jan 4 at 17:27
















          4














          Some trainers in ML.NET support training with an initial predictor, which means you can use an existing predictor as the starting point for training with new data.



          A test showing this can be found here with the relevant code being:



          // Train the first predictor.
          var trainer = ml.BinaryClassification.Trainers.StochasticDualCoordinateAscent("Label", "Features",advancedSettings: s => s.NumThreads = 1);
          var firstModel = trainer.Fit(trainData);

          // Train the second predictor on the same data.
          var secondTrainer = ml.BinaryClassification.Trainers.AveragedPerceptron("Label","Features");

          var trainRoles = new RoleMappedData(trainData, label: "Label", feature: "Features");
          var finalModel = secondTrainer.Train(new TrainContext(trainRoles, initialPredictor: firstModel.Model));





          share|improve this answer
























          • I think the OP is interested in adding more data to the first predictor, not creating a second type of predictor. I'm interested in this as well.

            – Webjedi
            Jan 4 at 15:55











          • Is it as simple as train model1 and model2 and then do finalmodel=model1.append(model2)? If both models are trained the same way?

            – Webjedi
            Jan 4 at 17:27














          4












          4








          4







          Some trainers in ML.NET support training with an initial predictor, which means you can use an existing predictor as the starting point for training with new data.



          A test showing this can be found here with the relevant code being:



          // Train the first predictor.
          var trainer = ml.BinaryClassification.Trainers.StochasticDualCoordinateAscent("Label", "Features",advancedSettings: s => s.NumThreads = 1);
          var firstModel = trainer.Fit(trainData);

          // Train the second predictor on the same data.
          var secondTrainer = ml.BinaryClassification.Trainers.AveragedPerceptron("Label","Features");

          var trainRoles = new RoleMappedData(trainData, label: "Label", feature: "Features");
          var finalModel = secondTrainer.Train(new TrainContext(trainRoles, initialPredictor: firstModel.Model));





          share|improve this answer













          Some trainers in ML.NET support training with an initial predictor, which means you can use an existing predictor as the starting point for training with new data.



          A test showing this can be found here with the relevant code being:



          // Train the first predictor.
          var trainer = ml.BinaryClassification.Trainers.StochasticDualCoordinateAscent("Label", "Features",advancedSettings: s => s.NumThreads = 1);
          var firstModel = trainer.Fit(trainData);

          // Train the second predictor on the same data.
          var secondTrainer = ml.BinaryClassification.Trainers.AveragedPerceptron("Label","Features");

          var trainRoles = new RoleMappedData(trainData, label: "Label", feature: "Features");
          var finalModel = secondTrainer.Train(new TrainContext(trainRoles, initialPredictor: firstModel.Model));






          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Nov 15 '18 at 22:34









          Gal OshriGal Oshri

          3182




          3182













          • I think the OP is interested in adding more data to the first predictor, not creating a second type of predictor. I'm interested in this as well.

            – Webjedi
            Jan 4 at 15:55











          • Is it as simple as train model1 and model2 and then do finalmodel=model1.append(model2)? If both models are trained the same way?

            – Webjedi
            Jan 4 at 17:27



















          • I think the OP is interested in adding more data to the first predictor, not creating a second type of predictor. I'm interested in this as well.

            – Webjedi
            Jan 4 at 15:55











          • Is it as simple as train model1 and model2 and then do finalmodel=model1.append(model2)? If both models are trained the same way?

            – Webjedi
            Jan 4 at 17:27

















          I think the OP is interested in adding more data to the first predictor, not creating a second type of predictor. I'm interested in this as well.

          – Webjedi
          Jan 4 at 15:55





          I think the OP is interested in adding more data to the first predictor, not creating a second type of predictor. I'm interested in this as well.

          – Webjedi
          Jan 4 at 15:55













          Is it as simple as train model1 and model2 and then do finalmodel=model1.append(model2)? If both models are trained the same way?

          – Webjedi
          Jan 4 at 17:27





          Is it as simple as train model1 and model2 and then do finalmodel=model1.append(model2)? If both models are trained the same way?

          – Webjedi
          Jan 4 at 17:27




















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53318863%2fml-net-add-new-data-to-exist-generated-model%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Bressuire

          Vorschmack

          Quarantine