add fra tasks#11
Open
AleksandrAstakhov wants to merge 3 commits into
Open
Conversation
Add French MTEB tasks for the DeepPavlov dialogue/intent datasets that had no fra subset yet: ViraIntent, Clinc150, DailyDialog (act/emotion), AirDialogue, Mantis, AbgCosQA, ClarQA, Canard, WizardOfWikipedia, Coral, FaithDial and TopiOCQA.
Samoed
reviewed
Jul 14, 2026
Comment on lines
+31
to
+38
| bibtex_citation=r""" | ||
| @inproceedings{guo2021abg, | ||
| author = {Guo, Meiqi and Zhang, Mingda and Reddy, Siva and Alikhani, Malihe}, | ||
| booktitle = {3rd Conference on Automated Knowledge Base Construction}, | ||
| title = {Abg-coqa: Clarifying ambiguity in conversational question answering}, | ||
| year = {2021}, | ||
| } | ||
| """, |
Member
There was a problem hiding this comment.
Лучше пока не добавлять цитаты, тк это мы сделали перевод
Fill in missing required TaskMetadata fields (reference, date, license) that caused test_all_metadata_is_filled_and_valid to fail for FrAbgCosQA, FrClarQA, FrCanard, FrMantisClassification and FrWizardOfWikipedia, using each task's own DeepPavlov dataset page as reference instead of the original English dataset's. Clear bibtex_citation on all fra tasks since these are DeepPavlov's French translations, not the cited work itself. Also fixes FrCoral's reference, which pointed at an unrelated audio dataset, and FrWizardOfWikipedia's bibtex, which was a copy-paste of Canard's.
Add FrWebLINXCandidatesReranking for DeepPavlov/weblinx_fr, the last available (non-blocked) French dataset from the extended dataset list. Uses AbsTaskReranking like RuWebLINXCandidatesReranking; no dataset_transform needed since the "query" column already holds the French text directly (query_dict keeps the original as a side field).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add French MTEB tasks for the DeepPavlov dialogue/intent datasets that had no fra subset yet: ViraIntent, Clinc150, DailyDialog (act/emotion), AirDialogue, Mantis, AbgCosQA, ClarQA, Canard, WizardOfWikipedia, Coral, FaithDial and TopiOCQA.
If you add a model or a dataset, please add the corresponding checklist: