Skip to content

Adding Korean and Hindi data#175

Open
raghavm243512 wants to merge 1 commit into
pr/medical_multilangfrom
pr/hi-ko
Open

Adding Korean and Hindi data#175
raghavm243512 wants to merge 1 commit into
pr/medical_multilangfrom
pr/hi-ko

Conversation

@raghavm243512

@raghavm243512 raghavm243512 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

Add Hindi and Korean to all domains

Helpful info for review:
WER configs will have some things stripped, this is because we are treating the characters as being "normalized". Flag this if the stripping would make the comparison totally wrong, otherwise it is fine, eg: pequeno == pequeño is fine for transcription comparison so that stripping is ok.
For medical hr, things will be a bit Us centric so entities or form names will often not be translated and may be slightly less natural, allow a small amount of flexibility here because many of he specific entities are almost never talked about in purely in hi/ko

@raghavm243512 raghavm243512 force-pushed the pr/medical_multilang branch from d23c696 to 986302c Compare July 2, 2026 21:26
@fanny-riols fanny-riols changed the title Pr/hi ko Adding Korean and Hindi data Jul 3, 2026
Comment thread data/airline_dataset.json
"de": "Hallo, ich muss meinen Flug auf den 25. März umbuchen."
"de": "Hallo, ich muss meinen Flug auf den 25. März umbuchen.",
"hi": "नमस्ते, मुझे अपनी flight March 25 के लिए बदलनी है।",
"ko": "안녕하세요, 제 항공편을 March 25로 변경해야 해요."

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"ko": "안녕하세요, 제 항공편을 March 25로 변경해야 해요."
"ko": "안녕하세요, 제 항공편을 3월 25일로 변경해야 해요."

The date was not translated.

Comment thread data/airline_dataset.json
"de": "Hallo, ich rufe wegen meines verspäteten Flugs an – können Sie mir die neusten Updates nennen?"
"de": "Hallo, ich rufe wegen meines verspäteten Flugs an – können Sie mir die neusten Updates nennen?",
"hi": "नमस्ते, मैं अपनी delayed flight के बारे में call कर रहा/रही हूँ—क्या आप मुझे updated details बता सकते हैं?",
"ko": "안녕하세요, 지연된 제 항공편 때문에 전화드렸는데요—업데이트된 정보를 알려 주실 수 있나요?"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"ko": "안녕하세요, 지연된 제 항공편 때문에 전화드렸는데요—업데이트된 정보를 알려 주실 수 있나요?"
"ko": "안녕하세요, 항공편이 지연되서 전화드렸는데요—업데이트된 정보를 알려 주실 수 있나요?"

Improved naturalness

Comment thread data/airline_dataset.json
"de": "Hallo – meine Flugzeit hat sich geändert und ich brauche Hilfe, um alles zu bestätigen."
"de": "Hallo – meine Flugzeit hat sich geändert und ich brauche Hilfe, um alles zu bestätigen.",
"hi": "नमस्ते—मेरी flight time बदल गई है और मुझे सब कुछ confirm करने में मदद चाहिए।",
"ko": "안녕하세요—제 항공편 시간이 변경돼서 모든 게 확정됐는지 확인하는 데 도움이 필요해요."

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"ko": "안녕하세요—제 항공편 시간이 변경돼서 모든 게 확정됐는지 확인하는 데 도움이 필요해요."
"ko": "안녕하세요—제 항공편 시간이 변경돼서 예약이 제대로 변경됐는지 확인하고 싶어요."

Improved naturalness

Comment thread data/airline_dataset.json
"de": "Hallo – können Sie mir sagen, was mit meinem Flug los ist, der nach Minneapolis zurückgekehrt ist? Wir sind planmäßig gegen 12:30 gestartet und mussten dann umdrehen."
"de": "Hallo – können Sie mir sagen, was mit meinem Flug los ist, der nach Minneapolis zurückgekehrt ist? Wir sind planmäßig gegen 12:30 gestartet und mussten dann umdrehen.",
"hi": "नमस्ते—क्या आप बता सकते हैं कि मेरी उस flight के साथ क्या हो रहा है जो Minneapolis वापस आ गई? हमने scheduled time पर, करीब 12:30 बजे take off किया था, और फिर वापस मुड़ना पड़ा",
"ko": "안녕하세요—Minneapolis로 되돌아온 제 항공편이 어떻게 된 건지 알려 주실 수 있나요? 예정 시간인 12:30쯤 이륙했는데, 그다음에 회항해야 했어요."

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"ko": "안녕하세요—Minneapolis로 되돌아온 제 항공편이 어떻게 된 건지 알려 주실 수 있나요? 예정 시간인 12:30쯤 이륙했는데, 그다음에 회항해야 했어요."
"ko": "안녕하세요—제 항공편이 Minneapolis로 회항했는데 어떻게 된 건지 알려 주실 수 있나요? 예정 시간인 12:30쯤 이륙했는데, 이후에 회항했어요."

Improved naturalness

Comment thread data/airline_dataset.json
"de": "Mein Flug wurde gestrichen und die Umbuchung, die man mir gegeben hat, ist nicht akzeptabel – können Sie mir helfen?"
"de": "Mein Flug wurde gestrichen und die Umbuchung, die man mir gegeben hat, ist nicht akzeptabel – können Sie mir helfen?",
"hi": "मेरी flight cancel हो गई है और जो rebooking उन्होंने मुझे दी है वह acceptable नहीं है—क्या आप मदद कर सकते हैं?",
"ko": "제 항공편이 취소됐고, 제공받은 재예약 일정이 받아들일 수 없는데요—도와주실 수 있나요?"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"ko": "제 항공편이 취소됐고, 제공받은 재예약 일정이 받아들일 수 없는데요—도와주실 수 있나요?"
"ko": "제 항공편이 취소됐는데, 새로 예약해 준 일정으로는 어려울 것 같아요.—도와주실 수 있을까요?"

Improved naturalness

Comment thread data/itsm_dataset.json
"de": "Hallo, ich brauche Hilfe, um einen externen Monitor für meinen Arbeitsplatz zu bekommen."
"de": "Hallo, ich brauche Hilfe, um einen externen Monitor für meinen Arbeitsplatz zu bekommen.",
"hi": "नमस्ते, मुझे अपने workstation के लिए external monitor लेने में मदद चाहिए।",
"ko": "안녕하세요, 제 workstation에 사용할 external monitor를 받는 데 도움이 필요해요."

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"ko": "안녕하세요, 제 workstation에 사용할 external monitor를 받는 데 도움이 필요해요."
"ko": "안녕하세요, 제 워크스테이션에 사용할 외부 모니터를 받는 데 도움이 필요해요."

Comment thread data/itsm_dataset.json
"de": "Hallo, ich brauche eine IntelliJ-Lizenz für meine Entwicklungsarbeit."
"de": "Hallo, ich brauche eine IntelliJ-Lizenz für meine Entwicklungsarbeit.",
"hi": "नमस्ते, मुझे अपने development work के लिए IntelliJ license लेना है।",
"ko": "안녕하세요, 개발 업무용 IntelliJ license가 필요해요."

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"ko": "안녕하세요, 개발 업무용 IntelliJ license가 필요해요."
"ko": "안녕하세요, 개발 업무용 IntelliJ 라이선스가 필요해요."

Comment thread data/itsm_dataset.json
"de": "Hallo, ich brauche eine temporäre Figma-Lizenz für ein Projekt."
"de": "Hallo, ich brauche eine temporäre Figma-Lizenz für ein Projekt.",
"hi": "नमस्ते, मुझे एक project के लिए temporary Figma license चाहिए।",
"ko": "안녕하세요, 프로젝트용 임시 Figma license가 필요해요."

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"ko": "안녕하세요, 프로젝트용 임시 Figma license가 필요해요."
"ko": "안녕하세요, 프로젝트용 임시 Figma 라이선스가 필요해요."

Comment thread data/itsm_dataset.json
"de": "Hallo, ich muss meine IntelliJ IDEA-Lizenz verlängern."
"de": "Hallo, ich muss meine IntelliJ IDEA-Lizenz verlängern.",
"hi": "नमस्ते, मुझे अपना IntelliJ IDEA license renew करना है।",
"ko": "안녕하세요, IntelliJ IDEA license를 갱신해야 해요."

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"ko": "안녕하세요, IntelliJ IDEA license를 갱신해야 해요."
"ko": "안녕하세요, IntelliJ IDEA 라이선스를 갱신해야 해요."

Comment thread data/itsm_dataset.json
"de": "Hallo, ich brauche Hilfe beim Zurücksetzen meines VPN-Passworts."
"de": "Hallo, ich brauche Hilfe beim Zurücksetzen meines VPN-Passworts.",
"hi": "नमस्ते, मुझे अपना VPN password reset करने में मदद चाहिए।",
"ko": "안녕하세요, VPN password 재설정하는 데 도움이 필요해요."

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"ko": "안녕하세요, VPN password 재설정하는 데 도움이 필요해요."
"ko": "안녕하세요, VPN 비밀번호 재설정하는 데 도움이 필요해요."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants