fix wikidata field broken for some languages

[k-yle]

today is an amazing day.... for years I thought the wikidata field was a dumb readonly field, and I've always used the raw tag editor to set this key.

I only realized today that it's supposed to have dropdown suggestions.... It's never worked for me, because my language is set to en-NZ which the wikidata API rejects, causing the field to delete everything you type into it

I'm not sure if simply stripping out the -XX country code suffix is the right solution, but this is an incredible QOL improvement

(although the suggestions are a bit weird)

[1ec5]

I'm not sure if simply stripping out the -XX country code suffix is the right solution, but this is an incredible QOL improvement

Unfortunately, this would regress important behavior in several languages, such as Serbian and Chinese, where the country or script would be quite significant. Even in English, Wikibase supports en-ca, en-gb, and en-us but not en-nz for some reason. This API request returns the list of languages supported for labels in Wikibase.

Considering that NZ English has some terms that differ from both British and American English, it may make sense to request it as a new Wikibase language on Phabricator, where a contributor recently worked out how to add new languages.

Actually, MediaWiki requires the language codes to be fully lowercase, not with uppercase regions as is the norm. iD has a workaround for American English that appears to be unnecessary, but I guess you could use it as a simple workaround for NZ English:

iD/modules/services/wikidata.js

Lines 90 to 94 in bd98ff9

	}).filter(function(code) {
	// HACK: en-us isn't a wikidata language. We should really be filtering by
	// the languages known to be supported by wikidata.
	return code !== 'en-us';
	});

[k-yle]

thanks, that's really helpful!

I had a look thru all the languages that iD supports, and of those, this issue seems to affect en-NZ, en-AU, en-IE, en-IN, nl-NL, he-IL, fa-IR, fr-FR, and ar-AA

I'm a bit confused though, I see this request to support en-IN and this one for en-AU appear to be "resolved", but neither locale is supported by the API...¹

Footnotes

1. I've been testing this by opening this URL in a new tab and tweaking language=* ↩

[1ec5]

I'm a bit confused though, I see this request to support en-IN and this one for en-AU appear to be "resolved", but neither locale is supported by the API...1

These were requests for Wikibase to accept new languages as the language of the value of a statement for which the property’s data type is “monolingual text”. For example, you can now add a statement to say this item’s name is something different in Indian English or that this item’s title is something different in Australian English, but this doesn’t affect either item’s label in any language.

Yes, MediaWiki/Wikibase have multiple differing lists of all the languages for hysterical reasons.

[1ec5]

This API query returns the list of interface languages supported by MediaWiki. However, MediaWiki is more lenient; you can use en-NZ as an interface language to get the expected results in English. (Using a more obscure interface language tends to prevent a site’s customized messages from coming through, but this is irrelevant for item queries.)

[tyrasd]

I solved this slightly differently in 862ca65 by just re-requesting the API with the simplified language code if the returned error looks like the language was the issue. This gracefully falls back to the intended behavior once the wikidata API adds supports for en-NZ, etc.

[1ec5]

Possibly relevant MediaWiki issue:

https://phabricator.wikimedia.org/T156220

[k-yle]

hey @tyrasd, your fix in 862ca65 doesn't seem to work for me, I think it might be because there's no return statement after this line, so the code is calling callback() immediately instead of retrying the request

[tyrasd]

Good catch! It was that in combination with the this parameter not being available in the retry code path. That's now fixed in ea054a0. Can you please double-check if it works?

[k-yle]

thanks! it works perfectly now