fix(rag): enforce chunk size and overlap when merging splits

[vineethsaivs]

Related issue

Fixes #7470

Summary

RAG text chunks repeat content with zero overlap and can exceed chunk_size; retained separators may also be inserted twice. Use the actual join separator for size accounting and discard retained splits until both the overlap and next-chunk size limits hold.

Verification

• Tests added for the changed behavior.
• uv run --no-sync pytest lib/crewai-tools/tests/rag -n 0 -q: 137 passed; four regressions fail before.
• Changed-source Ruff, formatting and mypy pass.

Additional context

Native Python 3.12 tests with network access blocked by the repository test configuration. No LLM calls or credentials required. AI assistance was used to investigate, implement and test this change.

Draft pending the required llm-generated label. GitHub returns 403 when this account attempts to add it; a maintainer needs to apply the label before review.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Vidit-Ostwal]

Ping me wheneever this is ready for reviiew

[vineethsaivs]

The patch and all 137 RAG tests are ready. @Vidit-Ostwal, could you add the required llm-generated label (GitHub returns 403 when I try)?