Repository navigation
De ITN: Fraction decoupled v2 - #525
adelina23dunina wants to merge 7 commits into
Conversation
Signed-off-by: Adelina Dunina <adunina@nvidia.com>
The Sparrowhawk Fraction message has no quantity field, so a fraction with a magnitude word failed to parse and the string stayed unnormalised. None of the other fraction grammars in the repository carry that field either. The non-deterministic magnitude abbreviations went with it, which leaves the tagger with no use for the deterministic flag. The space between two fields is now read from the input with NEMO_SPACE instead of being inserted and deleted separately, the way the Spanish tagger does it. The graph the other classes consume is renamed to graph_fraction_raw. Signed-off-by: Adelina Dunina <adunina@nvidia.com>
Signed-off-by: Adelina Dunina <adunina@nvidia.com>
Signed-off-by: Adelina Dunina <adunina@nvidia.com>
Signed-off-by: Adelina Dunina <adunina@nvidia.com>
Signed-off-by: Adelina Dunina <adunina@nvidia.com>
|
Hi @folivoramanh ! Could you please review it when you get a chance? Thank you! |
| eins 1 | ||
| ein 1 | ||
| einer 1 | ||
| einen 1 |
There was a problem hiding this comment.
is this missing and newly add?
There was a problem hiding this comment.
Yes, I noticed that some case endings for "ein" (one) were missing. Everything is fine now, these new additions also make it possible to get rid of hardcoded elements in fraction
| vier drei viertel~4 3/4 | ||
| minus vier drei viertel~-4 3/4 | ||
| ein millionstel~1/1000000 | ||
| das viertel ist schön~das viertel ist schön |
There was a problem hiding this comment.
if it's not fraction case, pls remove
There was a problem hiding this comment.
Hi Anh, thank you for the review! It was explained in the docstring but I removed it in order to follow the other languages pattern. I use "das viertel ist schön~das viertel ist schön" as a guard test: input equals output, so it pins the rule that a fraction requires a numerator. A bare denominator is not converted (viertel - 1/4), because Viertel is also an ordinary noun — a city district — so an implicit numerator of 1 would turn "das Viertel ist schön" (the neighborhood is nice) into "das 1/4 ist schön" (so fractions without a numerator are not converted)
Signed-off-by: Adelina Dunina <adunina@nvidia.com>
What does this PR do ?
Decouples the German ITN fraction tagger and verbalizer from the TN grammar.
The numerator and the denominator are now built from the ITN cardinal, the denominator stem comes from the ITN ordinal, and the data they need lives in
data/fractions/(half.tsv,suffixes.tsv) anddata/numbers/digits.tsv. The verbalizer is new on the ITN side and is registered inverbalize.py, so the class is verbalized by its own grammar instead of the TN one.taggers/ordinal.pychanges by four lines only: the ordinal stem is exposed asself.ordinal_stemso the fraction denominator can reuse it instead of rebuilding the table.Two commits expose graphs that have no consumer in this PR yet,
self.graph_wo_negativein the tagger andself.numbersin the verbalizer. They are what the measure grammar will consume in the next step, in the same spirit asself.graph_ordinalsin the ordinal tagger.Before your PR is "Ready for review"
Pre checks:
git commit -sto sign.pytestor (if your machine does not have GPU)pytest --cpufrom the root folder (given you marked your test cases accordingly@pytest.mark.run_only_on('CPU')).bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...pytestand Sparrowhawk here.__init__.pyfor every folder and subfolder, includingdatafolder which has .TSV files?Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.to all newly added Python files?Copyright 2015 and onwards Google, Inc.. See an example here.try import: ... except: ...) if not already done.PR Type: