Japanese letters get garbled in error messages. #2570

shusuke-nakamura · 2018-10-05T06:26:22Z

We are using rspec - core - 3.7.1, and tring to test files which encodings are set as 'CP932'.
When we run failing tests, Japanese letters in error messages get garbled.

For instance, if we run the test code below.

it "テスト実行結果の文字列には#{check_result}のメッセージが出力されること" do
  @msg.include?("【テスト実行結果】...#{check_result}".encode("CP932")).should be_truthy
end

The excution result would look like this.

Failures:

1) CheckTable#check_character 整合性チェックOKの場合 behaves like examples_check_character テスト実行 結果の文字列にはOKのメッセージが出力されること
Failure/Error: @msg.include?("縲舌ユ繧ｹ繝亥ｮ溯｡檎ｵ先棡縲?...#{check_result}".encode("CP932")).should be_truthy

expected: truthy value
got: false

In the error message, 【テスト実行結果】 became '縲舌ユ繧ｹ繝亥ｮ溯｡檎ｵ先棡縲?'.

Is there anyway to fix this issue?

The text was updated successfully, but these errors were encountered:

JonRowe · 2018-10-05T14:31:54Z

Are you able to pass the test? (e.g is it the expectation code thats wrong too? or just the differ?)

shusuke-nakamura · 2018-10-09T01:44:11Z

Thanks for the quick response.
Yes, we are able to pass the test. As I mentioned, this incident only happens in the error messages.

benoittgt · 2018-10-09T08:22:49Z

Can you provide us a reproduction script? You can look at https://github.com/rspec/rspec-core/blob/master/REPORT_TEMPLATE.md

shusuke-nakamura · 2018-10-10T01:15:36Z

Here is the reproduction script.

#-*- coding: CP932 -*-
# frozen_string_literal: true

begin
  require "bundler/inline"
rescue LoadError => e
  $stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler"
  raise e
end

gemfile(true) do
  source "https://rubygems.org"

  gem "rspec", "3.7.0" # Activate the gem and version you are reporting the issue against.
end

puts "Ruby version is: #{RUBY_VERSION}"

require 'rspec/autorun'

RSpec.describe 'Output' do
  it "テスト実行結果の文字列にはメッセージが出力されること" do
    msg = ""
    msg.include?("テスト実行結果".encode("CP932")).should be_truthy
  end
end

The result of the code above looks like this.

Ruby version is: 2.4.0
F

Failures:

  1) Output テスト実行結果の文字列にはメッセージが出力されること
     Failure/Error: msg.include?("?e?X?g???s????".encode("CP932")).should be_truthy

       expected: truthy value
            got: false
     # rspec_report.rb:10:in `block (2 levels) in <main>'

If you pass the expected value which is テスト実行結果 to the local variable msg, The result of the test would be successful

benoittgt · 2018-10-16T20:32:01Z

Hello @shusuke-nakamura

I'm not a professional of encoding but I have a different error when I try to run your script on ruby-2.4.4 and ruby-2.5.1.

$ ruby repro_script.rb
Fetching gem metadata from https://rubygems.org/..........
Resolving dependencies...
Using bundler 1.16.2
Fetching diff-lcs 1.3
Fetching rspec-support 3.7.1
Installing rspec-support 3.7.1
Installing diff-lcs 1.3
Fetching rspec-core 3.7.1
Fetching rspec-expectations 3.7.0
Fetching rspec-mocks 3.7.0
Installing rspec-core 3.7.1
Installing rspec-mocks 3.7.0
Installing rspec-expectations 3.7.0
Fetching rspec 3.7.0
Installing rspec 3.7.0
Ruby version is: 2.4.4
F

/Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/formatters/exception_presenter.rb:79:in `join': incompatible character encodings: Windows-31J and UTF-8 (Encoding::CompatibilityError)
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/formatters/exception_presenter.rb:79:in `fully_formatted'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/notifications.rb:200:in `fully_formatted'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/notifications.rb:114:in `block in fully_formatted_failed_examples'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/notifications.rb:113:in `each'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/notifications.rb:113:in `each_with_index'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/notifications.rb:113:in `fully_formatted_failed_examples'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/formatters/base_text_formatter.rb:32:in `dump_failures'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:206:in `block in notify'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:205:in `each'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:205:in `notify'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:175:in `block in finish'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:191:in `close_after'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:171:in `finish'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:81:in `report'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:112:in `run_specs'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:87:in `run'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:71:in `run'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:45:in `invoke'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:38:in `perform_at_exit'
	from /Users/bti/.rvm/gems/ruby-2.4.4/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:24:in `block in autorun'

Who link to another RSpec issue: rspec/rspec#28
Also: elizagamedev/wolftrans#2

Maybe looking at this one can help: https://tickets.puppetlabs.com/browse/PUP-8368

I will look at rspec/rspec#28 first then go back on this one.

Side note. Are you using Windows?

benoittgt · 2018-10-16T20:59:58Z

I'm able to reproduce with:

--- a/lib/rspec/core/formatters/exception_presenter.rb
+++ b/lib/rspec/core/formatters/exception_presenter.rb
@@ -1,4 +1,4 @@
-# encoding: utf-8
+# encoding: CP932

Failures:

  1) Output テスト実行結果の文字列にはメッセージが出力されること
     Failure/Error: msg.include?("�e�X�g���s����".encode("CP932")).should be_truthy

     NameError:
       undefined local variable or method `be_truthy' for #<RSpec::ExampleGroups::Output:0x00007f91cecc0a30>
     # repro_script.rb:25:in `block (2 levels) in <main>'

Finished in 0.00358 seconds (files took 0.04909 seconds to load)
1 example, 1 failure

Failed examples:

rspec repro_script.rb:23 # Output テスト実行結果の文字列にはメッセージが出力されること

shusuke-nakamura · 2018-10-18T02:10:52Z

@benoittgt
Thank you for your response.
Yes, I am using windows.
I appreciate your further research on this matter.

When the user create a test with CP932 encoding and japanese chars RSpec fail to properly display characters or crash with the error: > Encoding::CompatibilityError: incompatible character encodings: Windows-31J and UTF-8 Fix: - rspec#2570 - https://github.com/rspec/rspec-core/issues/2543

When the user create a test with CP932 encoding and japanese chars RSpec fail to properly display characters or crash with the error: > Encoding::CompatibilityError: incompatible character encodings: Windows-31J and UTF-8 Fix: - rspec#2570 - https://github.com/rspec/rspec-core/issues/2543 For Ruby 1.8.7 We are following the same behavior as for rspec-support https://github.com/rspec/rspec-support/blob/9940a8656071655b807f772f36101b4d27f1b67d/lib/rspec/support/spec/string_matcher.rb#L8

benoittgt · 2018-11-27T13:19:53Z

It should be fixed with #2575

Do you think you can try it @shusuke-nakamura?

We will reopen the issue if it is not fixed.

When the user create a test with CP932 encoding and japanese chars RSpec fail to properly display characters or crash with the error: > Encoding::CompatibilityError: incompatible character encodings: Windows-31J and UTF-8 Fix: - #2570 - https://github.com/rspec/rspec-core/issues/2543 For Ruby 1.8.7 We are following the same behavior as for rspec-support https://github.com/rspec/rspec-support/blob/9940a8656071655b807f772f36101b4d27f1b67d/lib/rspec/support/spec/string_matcher.rb#L8

When the user create a test with CP932 encoding and japanese chars RSpec fail to properly display characters or crash with the error: > Encoding::CompatibilityError: incompatible character encodings: Windows-31J and UTF-8 Fix: - rspec#2570 - https://github.com/rspec/rspec-core/issues/2543 For Ruby 1.8.7 We are following the same behavior as for rspec-support https://github.com/rspec/rspec-support/blob/9940a8656071655b807f772f36101b4d27f1b67d/lib/rspec/support/spec/string_matcher.rb#L8

…iles When the user create a test with CP932 encoding and japanese chars RSpec fail to properly display characters or crash with the error: > Encoding::CompatibilityError: incompatible character encodings: Windows-31J and UTF-8 Fix: - rspec/rspec-core#2570 - https://github.com/rspec/rspec-core/issues/2543 For Ruby 1.8.7 We are following the same behavior as for rspec-support https://github.com/rspec/rspec-support/blob/9940a8656071655b807f772f36101b4d27f1b67d/lib/rspec/support/spec/string_matcher.rb#L8 --- This commit was imported from rspec/rspec-core@fce1978.

…iles When the user create a test with CP932 encoding and japanese chars RSpec fail to properly display characters or crash with the error: > Encoding::CompatibilityError: incompatible character encodings: Windows-31J and UTF-8 Fix: - rspec/rspec-core#2570 - https://github.com/rspec/rspec-core/issues/2543 For Ruby 1.8.7 We are following the same behavior as for rspec-support https://github.com/rspec/rspec-support/blob/9940a8656071655b807f772f36101b4d27f1b67d/lib/rspec/support/spec/string_matcher.rb#L8 --- This commit was imported from rspec/rspec-core@05611c5.

benoittgt mentioned this issue Oct 16, 2018

If the test code is CP 932 and Japanese is included in failure / error, Encoding::CompatibilityError rspec/rspec#28

Open

benoittgt mentioned this issue Oct 19, 2018

Handle RSpec description with japanese char in CP932 encoded files #2575

Merged

benoittgt closed this as completed Nov 27, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Japanese letters get garbled in error messages. #2570

Japanese letters get garbled in error messages. #2570

shusuke-nakamura commented Oct 5, 2018 •

edited

Loading

JonRowe commented Oct 5, 2018

shusuke-nakamura commented Oct 9, 2018

benoittgt commented Oct 9, 2018

shusuke-nakamura commented Oct 10, 2018 •

edited

Loading

benoittgt commented Oct 16, 2018 •

edited

Loading

benoittgt commented Oct 16, 2018

shusuke-nakamura commented Oct 18, 2018

benoittgt commented Nov 27, 2018 •

edited

Loading

Japanese letters get garbled in error messages. #2570

Japanese letters get garbled in error messages. #2570

Comments

shusuke-nakamura commented Oct 5, 2018 • edited Loading

JonRowe commented Oct 5, 2018

shusuke-nakamura commented Oct 9, 2018

benoittgt commented Oct 9, 2018

shusuke-nakamura commented Oct 10, 2018 • edited Loading

benoittgt commented Oct 16, 2018 • edited Loading

benoittgt commented Oct 16, 2018

shusuke-nakamura commented Oct 18, 2018

benoittgt commented Nov 27, 2018 • edited Loading

shusuke-nakamura commented Oct 5, 2018 •

edited

Loading

shusuke-nakamura commented Oct 10, 2018 •

edited

Loading

benoittgt commented Oct 16, 2018 •

edited

Loading

benoittgt commented Nov 27, 2018 •

edited

Loading