How to handle exception and skip the wrong csv line as well?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
I am reading a csv file using Spring batch. I am reading the content of csv and writing this content to the database in accordance with one of the entity class.
Now there could be certain lines in csv that are wrong and don't match the POJO attributes. To handle this I configured my Step as follows:
Step step = stepBuilderFactory.get("CSV-Step")
.<Book, Book>chunk(100)
.faultTolerant()
.skip(FlatFileParseException.class)
.skipLimit(1)
.reader(itemReader)
.writer(itemWriter)
.build();
It basically skips the line which causes FlatFileParseException and goes on with subsequent lines. Now I also want to log the lines for which parsing could not be done. For this in my GlobalExceptionHandler annotated with @ControllerAdvice I made following method:
@OnReadError
public void handleCsvParseException(FlatFileParseException ex, Throwable throwable) {
logger.error("! FlatFileParseException, line is: " + ex.getLineNumber());
logger.error("! FlatFileParseException, input is: " + ex.getInput());
logger.error("! Message: " + throwable.getMessage());
logger.error("! Cause: " + throwable.getCause());
}
The thing is that this method is not being called because i have the skip configuration in my Step. How can i ignore the unwanted lines i.e. skipping unwanted line but at the same time log information about them. I would appreciate any kind of help.
spring csv exception-handling spring-batch
asked Nov 17 '18 at 1:21
Navjot SinghNavjot Singh
18910
add a comment |
I am reading a csv file using Spring batch. I am reading the content of csv and writing this content to the database in accordance with one of the entity class.
Now there could be certain lines in csv that are wrong and don't match the POJO attributes. To handle this I configured my Step as follows:
Step step = stepBuilderFactory.get("CSV-Step")
.<Book, Book>chunk(100)
.faultTolerant()
.skip(FlatFileParseException.class)
.skipLimit(1)
.reader(itemReader)
.writer(itemWriter)
.build();
It basically skips the line which causes FlatFileParseException and goes on with subsequent lines. Now I also want to log the lines for which parsing could not be done. For this in my GlobalExceptionHandler annotated with @ControllerAdvice I made following method:
@OnReadError
public void handleCsvParseException(FlatFileParseException ex, Throwable throwable) {
logger.error("! FlatFileParseException, line is: " + ex.getLineNumber());
logger.error("! FlatFileParseException, input is: " + ex.getInput());
logger.error("! Message: " + throwable.getMessage());
logger.error("! Cause: " + throwable.getCause());
}
The thing is that this method is not being called because i have the skip configuration in my Step. How can i ignore the unwanted lines i.e. skipping unwanted line but at the same time log information about them. I would appreciate any kind of help.
spring csv exception-handling spring-batch
asked Nov 17 '18 at 1:21
Navjot SinghNavjot Singh
18910
add a comment |
I am reading a csv file using Spring batch. I am reading the content of csv and writing this content to the database in accordance with one of the entity class.
Now there could be certain lines in csv that are wrong and don't match the POJO attributes. To handle this I configured my Step as follows:
Step step = stepBuilderFactory.get("CSV-Step")
.<Book, Book>chunk(100)
.faultTolerant()
.skip(FlatFileParseException.class)
.skipLimit(1)
.reader(itemReader)
.writer(itemWriter)
.build();
It basically skips the line which causes FlatFileParseException and goes on with subsequent lines. Now I also want to log the lines for which parsing could not be done. For this in my GlobalExceptionHandler annotated with @ControllerAdvice I made following method:
@OnReadError
public void handleCsvParseException(FlatFileParseException ex, Throwable throwable) {
logger.error("! FlatFileParseException, line is: " + ex.getLineNumber());
logger.error("! FlatFileParseException, input is: " + ex.getInput());
logger.error("! Message: " + throwable.getMessage());
logger.error("! Cause: " + throwable.getCause());
}
The thing is that this method is not being called because i have the skip configuration in my Step. How can i ignore the unwanted lines i.e. skipping unwanted line but at the same time log information about them. I would appreciate any kind of help.
spring csv exception-handling spring-batch
asked Nov 17 '18 at 1:21
Navjot SinghNavjot Singh
18910
I am reading a csv file using Spring batch. I am reading the content of csv and writing this content to the database in accordance with one of the entity class.
Now there could be certain lines in csv that are wrong and don't match the POJO attributes. To handle this I configured my Step as follows:
Step step = stepBuilderFactory.get("CSV-Step")
.<Book, Book>chunk(100)
.faultTolerant()
.skip(FlatFileParseException.class)
.skipLimit(1)
.reader(itemReader)
.writer(itemWriter)
.build();
It basically skips the line which causes FlatFileParseException and goes on with subsequent lines. Now I also want to log the lines for which parsing could not be done. For this in my GlobalExceptionHandler annotated with @ControllerAdvice I made following method:
@OnReadError
public void handleCsvParseException(FlatFileParseException ex, Throwable throwable) {
logger.error("! FlatFileParseException, line is: " + ex.getLineNumber());
logger.error("! FlatFileParseException, input is: " + ex.getInput());
logger.error("! Message: " + throwable.getMessage());
logger.error("! Cause: " + throwable.getCause());
}
The thing is that this method is not being called because i have the skip configuration in my Step. How can i ignore the unwanted lines i.e. skipping unwanted line but at the same time log information about them. I would appreciate any kind of help.
spring csv exception-handling spring-batch
spring csv exception-handling spring-batch
asked Nov 17 '18 at 1:21
Navjot SinghNavjot Singh
18910
asked Nov 17 '18 at 1:21
Navjot SinghNavjot Singh
18910
asked Nov 17 '18 at 1:21
Navjot SinghNavjot Singh
18910
asked Nov 17 '18 at 1:21
Navjot SinghNavjot Singh
18910
asked Nov 17 '18 at 1:21
Navjot SinghNavjot Singh
18910
18910
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
The SkipListener is what you need. This listener will be called whenever the configured skippable exception occurs during reading, processing or writing items.
In your case, you can implement the logging logic in the SkipListener#onSkipInRead(Throwable t); method. The FlatFileParseException will be passed as a parameter and will give you the necessary context (the line number and the raw input).
Hope this helps.
answered Nov 19 '18 at 7:58
Mahmoud Ben HassineMahmoud Ben Hassine
5,7751717
yes that makes sense
– Navjot Singh
Nov 19 '18 at 10:11
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53347349%2fhow-to-handle-exception-and-skip-the-wrong-csv-line-as-well%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The SkipListener is what you need. This listener will be called whenever the configured skippable exception occurs during reading, processing or writing items.
In your case, you can implement the logging logic in the SkipListener#onSkipInRead(Throwable t); method. The FlatFileParseException will be passed as a parameter and will give you the necessary context (the line number and the raw input).
Hope this helps.
answered Nov 19 '18 at 7:58
Mahmoud Ben HassineMahmoud Ben Hassine
5,7751717
yes that makes sense
– Navjot Singh
Nov 19 '18 at 10:11
add a comment |
The SkipListener is what you need. This listener will be called whenever the configured skippable exception occurs during reading, processing or writing items.
In your case, you can implement the logging logic in the SkipListener#onSkipInRead(Throwable t); method. The FlatFileParseException will be passed as a parameter and will give you the necessary context (the line number and the raw input).
Hope this helps.
answered Nov 19 '18 at 7:58
Mahmoud Ben HassineMahmoud Ben Hassine
5,7751717
yes that makes sense
– Navjot Singh
Nov 19 '18 at 10:11
add a comment |
The SkipListener is what you need. This listener will be called whenever the configured skippable exception occurs during reading, processing or writing items.
In your case, you can implement the logging logic in the SkipListener#onSkipInRead(Throwable t); method. The FlatFileParseException will be passed as a parameter and will give you the necessary context (the line number and the raw input).
Hope this helps.
answered Nov 19 '18 at 7:58
Mahmoud Ben HassineMahmoud Ben Hassine
5,7751717
The SkipListener is what you need. This listener will be called whenever the configured skippable exception occurs during reading, processing or writing items.
In your case, you can implement the logging logic in the SkipListener#onSkipInRead(Throwable t); method. The FlatFileParseException will be passed as a parameter and will give you the necessary context (the line number and the raw input).
Hope this helps.
answered Nov 19 '18 at 7:58
Mahmoud Ben HassineMahmoud Ben Hassine
5,7751717
answered Nov 19 '18 at 7:58
Mahmoud Ben HassineMahmoud Ben Hassine
5,7751717
answered Nov 19 '18 at 7:58
Mahmoud Ben HassineMahmoud Ben Hassine
5,7751717
answered Nov 19 '18 at 7:58
Mahmoud Ben HassineMahmoud Ben Hassine
5,7751717
5,7751717
yes that makes sense
– Navjot Singh
Nov 19 '18 at 10:11
add a comment |
yes that makes sense
– Navjot Singh
Nov 19 '18 at 10:11
yes that makes sense
– Navjot Singh
Nov 19 '18 at 10:11
yes that makes sense
– Navjot Singh
Nov 19 '18 at 10:11
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53347349%2fhow-to-handle-exception-and-skip-the-wrong-csv-line-as-well%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
