I have been told that the "tagging-parsed" endpoint is merely an expansion of the "tagging" endpoint of Dharmamitra API, providing additional information while relying on the same underlying model. If both APIs indeed use the same model, it is unclear why their outputs differ significantly. In practice, the "tagging" API often returns incorrect results compared to "tagging-parsed".
For example, in the following case, the words लहरि, वीचि, and मयाम् returned by the "tagging" API appear to be incorrect:
```
curl -X POST https://dharmamitra.org/api-tagging/tagging/ \
-H "Content-Type: application/json" \
-d '{"texts":["mṛdulasmitāṃśulaharījyotsnāvayasyānvitāmārohatyapavargasauṃdhavalabhīmānandavīcīmayīm"], "grammar_type":"unsandhied-lemma-morphosyntax"}'
मृदुल स्मित अंशु लहरि ज्योत्स्ना वयस्य अन्विताम् आरोहति अपवर्ग सौंधवल भीम आनन्द वीचि मयाम्
```
However, the "tagging-parsed" endpoint produces a more accurate result:
```
curl -X POST https://dharmamitra.org/api-tagging/tagging-parsed/ \
-H "Content-Type: application/json" \
-d '{"texts":["mṛdulasmitāṃśulaharījyotsnāvayasyānvitāmārohatyapavargasauṃdhavalabhīmānandavīcīmayīm"], "grammar_type":"western"}'
मृदुल स्मित अंशु लहरी ज्योत्स्ना वयस्य अन्विताम् आरोहति अपवर्ग सौंधवल भीम आनन्द वीची मयीम्
```
If the implication is that "tagging-parsed" is suitable for my use case then another issue arises: in several cases it returns no result at all. For instance:
```
curl -X POST https://dharmamitra.org/api-tagging/tagging-parsed/ \
-H "Content-Type: application/json" \
-d '{"texts":["mṛdulasmitāṃśulaharījyotsnā"], "grammar_type":"western"}'
```
This request returns no output.
The main developer is already aware of this issue:
https://github.com/sebastian-nehrdich/byt5-sanskrit-analyzers/issues/2
There are many such instances where "tagging-parsed" produces no result. In these situations, switching to the "tagging" API is not a reliable workaround, as it may return incorrect analyses. The alternative would be to fall back on the older Hellwig API, but that too is not entirely accurate. For example, it returns:
मृदुल स्मित अंशुल हरीज्योत्स्ना
which is semantically incorrect. The correct segmentation should be:
मृदुल स्मित अंशु लहरी ज्योत्स्ना
Interestingly, the older data-driven website "Sansadhani" provides the correct result in this case. This means that I currently need to rely on three different models to obtain reasonably reliable results. This situation seems difficult to reconcile with the claim "One Model Is All You Need: ByT5-Sanskrit"!
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.