Skip to content

[Pipe] Use buffered input for TsFile scan parser#18254

Open
Caideyipi wants to merge 2 commits into
apache:masterfrom
Caideyipi:pipe-buffered-tsfile-input
Open

[Pipe] Use buffered input for TsFile scan parser#18254
Caideyipi wants to merge 2 commits into
apache:masterfrom
Caideyipi:pipe-buffered-tsfile-input

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Use buffered input for Pipe TsFile scanning

  • Upgrade TsFile to 2.3.2-260720-SNAPSHOT, which includes BufferedTsFileInput.
  • Use BufferedTsFileInput for the no-modification path of TsFileInsertionEventScanParser, where the reader performs small sequential reads.
  • Keep the existing metadata-loading and device-metadata-cache behavior when modifications are present.

Preserve old TsFile compatibility

The TsFileInput-based TsFileSequenceReader constructor does not initialize the compatibility deserializer. The parser therefore checks the file version before transferring ownership of the buffered input and falls back to the original String-based constructor for older TsFile versions.

Validation

  • mvn spotless:check -pl iotdb-core/datanode
  • mvn checkstyle:check -pl iotdb-core/datanode
  • mvn dependency:tree -pl iotdb-core/datanode -Dincludes=org.apache.tsfile:* -Dverbose
  • DataNode javac reached all main sources without errors in the changed file; the module remains blocked by unrelated stale generated/i18n/Thrift symbols in the local workspace.

This PR has:

  • been self-reviewed.
  • added or updated version, license, or notice information
  • added comments explaining the why and the intent of the code wherever it would not be obvious for an unfamiliar reader.

Key changed/added classes (or packages if there are too many classes) in this PR
  • TsFileInsertionEventScanParser
  • Root pom.xml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant