Hello Spring Batch team,
Bug description
In Spring Batch 6.0.3, chunk scanning after rollback in writer appears to run in a single transaction instead of running each scanned item in its own transaction.
This appears to be a regression from Spring Batch 5.
Environment
- Spring Batch: 6.0.3
- Java: 17+
- Database: H2
- Build tool: Maven
Steps to reproduce
The example uses the following scenario:
Fours items (1,2,3,4) are written in the same chunk, a skip policy skips every exception, with a configurable skip limit.
Item 2 and 3 throw exceptions, after performing an insert in the database.
With skipLimit=1, the step fails and no item is committed. Expected: item 1 should be committed before items 2 and 3 make the step fail.
With skipLimit=2, items 2 and 3 are committed even though they threw exceptions, database contains all items 1,2,3,4. Expected: only items 1 and 4 should be commited.
@Bean
public ListItemReader<String> reader() {
return new ListItemReader<>(Arrays.asList("1", "2", "3", "4"));
}
@Bean
public ItemWriter<String> writer(JdbcTemplate jdbcTemplate) {
return items -> {
System.out.println(); System.out.println("Writing chunk of items..." + items); System.out.println();
for (String number : items) {
System.out.println("Writing item: " + number);
writeItem(jdbcTemplate, number);
System.out.println();
}
};
}
private void writeItem(JdbcTemplate jdbcTemplate, String number) {
// Write item to database
jdbcTemplate.execute("insert into delivery (item_number) values ('" + number + "')");
// Throw error if item 2 or 3
if (number.equals("2") || number.equals("3")) {
System.out.println("Simulating error on item: " + number);
throw new RuntimeException("Simulated write error for item: " + number);
}
}
@Bean
public Step step(
JobRepository jobRepository, PlatformTransactionManager tm,
ListItemReader<String> reader, ItemWriter<String> writer) {
return new StepBuilder("step", jobRepository)
.<String, String>chunk(4)
.transactionManager(tm)
.reader(reader)
.writer(writer)
.faultTolerant()
.skip(Exception.class)
.skipLimit(skipLimit)
.build();
}
- Build and run the Spring Batch 6 project.
- Run once with
--skipLimit=1 and once with --skipLimit=2.
- Command line to run the example:
mvn package exec:java -Dexec.mainClass=org.springframework.batch.MyBatchJobConfiguration -Dexec.args="--skipLimit=1"
mvn package exec:java -Dexec.mainClass=org.springframework.batch.MyBatchJobConfiguration -Dexec.args="--skipLimit=2"
- Observe the final database content printed at the end of the execution.
Output below refers to what is printed by:
System.out.println("*** Deliveries in database after job execution - Start ***");
jdbcTemplate.queryForStream("select * from delivery", new DataClassRowMapper<>(Delivery.class))
.forEach(delivery -> System.out.println("Delivery item number: " + delivery.itemNumber()));
System.out.println("*** Deliveries in database after job execution - End ***");
Expected behavior
During scan mode, each scanned item should be processed in its own transaction and committed/rolled back independently.
skipLimit=1: item 1 should remain committed before the step fails.
skipLimit=2: items 2 and 3 should be skipped (not committed), so only 1 and 4 should remain.
Minimal Complete Reproducible example
Two minimal Maven projects are provided:
- Spring Batch 5 project (reference)
- Spring Batch 6 project (issue)
Observed vs expected output:
Scenario A: skipLimit=1
Obtained:
*** Deliveries in database after job execution - Start ***
*** Deliveries in database after job execution - End ***
Expected:
*** Deliveries in database after job execution - Start ***
Delivery item number: 1
*** Deliveries in database after job execution - End ***
Scenario B: skipLimit=2
Obtained:
*** Deliveries in database after job execution - Start ***
Delivery item number: 1
Delivery item number: 2
Delivery item number: 3
Delivery item number: 4
*** Deliveries in database after job execution - End ***
Expected:
*** Deliveries in database after job execution - Start ***
Delivery item number: 1
Delivery item number: 4
*** Deliveries in database after job execution - End ***
spring-batch-5.zip
spring-batch-6.zip
Hello Spring Batch team,
Bug description
In Spring Batch 6.0.3, chunk scanning after rollback in writer appears to run in a single transaction instead of running each scanned item in its own transaction.
This appears to be a regression from Spring Batch 5.
Environment
Steps to reproduce
The example uses the following scenario:
Fours items (1,2,3,4) are written in the same chunk, a skip policy skips every exception, with a configurable skip limit.
Item 2 and 3 throw exceptions, after performing an insert in the database.
With
skipLimit=1, the step fails and no item is committed. Expected: item 1 should be committed before items 2 and 3 make the step fail.With
skipLimit=2, items 2 and 3 are committed even though they threw exceptions, database contains all items 1,2,3,4. Expected: only items 1 and 4 should be commited.--skipLimit=1and once with--skipLimit=2.Output below refers to what is printed by:
Expected behavior
During scan mode, each scanned item should be processed in its own transaction and committed/rolled back independently.
skipLimit=1: item 1 should remain committed before the step fails.skipLimit=2: items 2 and 3 should be skipped (not committed), so only 1 and 4 should remain.Minimal Complete Reproducible example
Two minimal Maven projects are provided:
Observed vs expected output:
Scenario A:
skipLimit=1Obtained:
Expected:
Scenario B:
skipLimit=2Obtained:
Expected:
spring-batch-5.zip
spring-batch-6.zip