Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ArgumentError: unknown encoding name - iso-8859-1|utf-8 #254

Closed
rstueven opened this issue Jul 14, 2022 · 2 comments
Closed

ArgumentError: unknown encoding name - iso-8859-1|utf-8 #254

rstueven opened this issue Jul 14, 2022 · 2 comments

Comments

@rstueven
Copy link

Ruby 2.7.6 / Rails 6.0.5

This line worked without error in Ruby 2.4.6:
csv_rows = CSV.parse(file_contents, headers: true, encoding: 'iso-8859-1|utf-8')

It fails with this error in Ruby 2.7.6:
ArgumentError: unknown encoding name - iso-8859-1|utf-8

I have also tried ISO-8859-1|UTF-8, iso-8859-1:utf-8, and ISO-8859-1:UTF-8, and they give the same error.

Both encodings exist, but it doesn't find the combination:

[3] pry(main)> Encoding.find("iso-8859-1|utf-8")
ArgumentError: unknown encoding name - iso-8859-1|utf-8
from (pry):3:in `find'
[4] pry(main)> Encoding.find("iso-8859-1")
=> #<Encoding:ISO-8859-1>
[5] pry(main)> Encoding.find("utf-8")
=> #<Encoding:UTF-8>
[6] pry(main)> Encoding.find("iso-8859-1:utf-8")
ArgumentError: unknown encoding name - iso-8859-1:utf-8
from (pry):7:in `find'

This was supposedly fixed in Issue #23 , but I'm still getting the error.

@kou kou closed this as completed in 3eeaeef Jul 15, 2022
@kou
Copy link
Member

kou commented Jul 15, 2022

In this case, encoding: isn't used in Ruby 2.4.6. file_contents.encoding is used.

And iso-8859-1|utf-8 is invalid syntax. You need to use : instead of |: iso-8859-1:utf-8

Anyway, it's a backward incompatible that csv in Ruby 2.7 raises an exception for iso-8859-1:utf-8. So I've added support for transcoding input data even when input data is String. Because csv transcodes input data with iso-8859-1:utf-8 when input data is IO.

In your case, could you use the following code? This will work with csv in old Ruby too.

csv_rows = CSV.parse(file_contents.encode("UTF-8", "ISO-8859-1"), headers: true)

@rstueven
Copy link
Author

In your case, could you use the following code? This will work with csv in old Ruby too.

csv_rows = CSV.parse(file_contents.encode("UTF-8", "ISO-8859-1"), headers: true)

It looks like that worked. Thanks!

peterzhu2118 pushed a commit to Shopify/ruby that referenced this issue Dec 6, 2022
GitHub: fix ruby/csv#254

Syntax is "from-encoding:to-encoding".

Reported by Richard Stueven. Thanks!!!

ruby/csv@3eeaeefc67
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants