Add roberta implementation for MLM using smile strings.#2122

Draft

tnat410 wants to merge 6 commits into

LBANN:v1.x-developfrom

tnat410:roberta_atom

Commits on Jun 27, 2022

Add roberta implementation for MLM using smile strings.
Tuan Nguyen Anh Tran
committed
Add re-tokenizing script.
Tuan Nguyen Anh Tran
committed

Commits on Jun 29, 2022

Updating masking function based on HuggingFace implementation.
Tuan Nguyen Anh Tran
committed

Commits on Jul 26, 2022

Fix weights initialization. Update datareader.
Tuan Nguyen Anh Tran
committed

Commits on Jul 27, 2022

Add evaluation script.
Tuan Nguyen Anh Tran
committed
Update model config.
Tuan Nguyen Anh Tran
committed