Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: replace github.com/ghodss/yaml with gopkg.in/yaml.v3 #230

Merged
merged 2 commits into from
Mar 11, 2023
Merged

refactor: replace github.com/ghodss/yaml with gopkg.in/yaml.v3 #230

merged 2 commits into from
Mar 11, 2023

Conversation

Juneezee
Copy link
Contributor

@Juneezee Juneezee commented Mar 10, 2023

The package github.com/ghodss/yaml is no longer actively maintained. See discussion in ghodss/yaml#75 and ghodss/yaml#80. sigs.k8s.io/yaml is a permanent fork of github.com/ghodss/yaml.

The notable change is that github.com/ghodss/yaml uses gopkg.in/yaml.v2 v2.2.2, but sigs.k8s.io/yaml uses gopkg.in/yaml.v2 v2.4.0. Changes can be seen here v2.2.2...v2.4.0, mostly bug fixes.

@cole-miller
Copy link
Contributor

cole-miller commented Mar 10, 2023

Thanks for the PR and for bringing up the maintenance status of github.com/ghodss/yaml. Since our YAML needs are so simple, perhaps we could get away with just using gopkg.in/yaml.v2 (or v3) directly and eliminating a dependency?

@cole-miller cole-miller self-assigned this Mar 10, 2023
@tomponline
Copy link
Member

Thanks for the PR and for bringing up the maintenance status of github.com/ghodss/yaml. Since our YAML needs are so simple, perhaps we could get away with just using gopkg.in/yaml.v2 (or v3) directly and eliminating a dependency?

That would be preferable.

@cole-miller
Copy link
Contributor

@Juneezee, would you be up for reworking this PR to use gopkg.in/yaml.v3?

At the time of making this commit, the package `github.com/ghodss/yaml`
is no longer actively maintained.

Signed-off-by: Eng Zer Jun <[email protected]>
@Juneezee Juneezee changed the title refactor: replace github.com/ghodss/yaml with sigs.k8s.io/yaml refactor: replace github.com/ghodss/yaml with gopkg.in/yaml.v3 Mar 11, 2023
@Juneezee
Copy link
Contributor Author

@Juneezee, would you be up for reworking this PR to use gopkg.in/yaml.v3?

Done!

Copy link
Contributor

@cole-miller cole-miller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just ran some tests locally and it seems that -- unlike github.com/ghodss/yaml -- gopkg.in/yaml.v3 does some case conversion by default:

package main

import (
	orig "github.com/ghodss/yaml"
	yamlv3 "gopkg.in/yaml.v3"
)

type NodeInfo struct {
	ID uint64
	Address string
	Role int
}

func main() {
	info := NodeInfo{ 5, "abc", 7 }
	data, _ := orig.Marshal(&info)
	println(string(data))
	data, _ = yamlv3.Marshal(&info)
	println(string(data))
}
$ go run
Address: abc
ID: 5
Role: 7

id: 5
address: abc
role: 7

Could you add yaml: annotations on this declaration so that the replacement will be backwards-compatible? (We want each new version of go-dqlite to be able to read node store and info files written by a previous version.)

type NodeInfo struct {
ID uint64
Address string
Role NodeRole
}

@Juneezee
Copy link
Contributor Author

I just ran some tests locally and it seems that -- unlike github.com/ghodss/yaml -- gopkg.in/yaml.v3 does some case conversion by default

Yup, I noticed the same too. The tests are failing with both gopkg.in/yaml.v2 and gopkg.in/yaml.v3 (https://github.com/Juneezee/go-dqlite/actions/runs/4390622459).

gopkg.in/yaml.v2 and gopkg.in/yaml.v3 marshal and unmarshal using the field name lowercased as default key if we don't explicitly specify a yaml field tag. (https://pkg.go.dev/gopkg.in/yaml.v3#Marshal)

Unlike `github.com/ghodss/yaml`, `gopkg.in/yaml.v3` marshals using the
field name lowercased as the default key [1].

[1]: https://pkg.go.dev/gopkg.in/yaml.v3#Marshal
Signed-off-by: Eng Zer Jun <[email protected]>
@Juneezee Juneezee requested a review from cole-miller March 11, 2023 06:26
Copy link
Contributor

@cole-miller cole-miller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@cole-miller cole-miller merged commit 394129c into canonical:master Mar 11, 2023
@MathieuBordere
Copy link

MathieuBordere commented Mar 13, 2023

seems to hit a segfault on s390x https://launchpadlibrarian.net/655817390/buildlog_ubuntu-focal-s390x.go-dqlite_1.11.7+git5-g394129c~focal1_BUILDING.txt.gz

--- FAIL: TestNew_JoinerRestart (0.28s)
    app_test.go:1073: 19:04:03.209 - 4: DEBUG: new connection from 127.0.0.1:38408
    app_test.go:1073: 19:04:03.212 - 4: DEBUG: attempt 1: server 127.0.0.1:9001: connected
    app_test.go:1073: 19:04:03.213 - 4: DEBUG: new connection from 127.0.0.1:38410
    app_test.go:1073: 19:04:03.306 - 4: DEBUG: new connection from 127.0.0.1:38426
    app_test.go:1073: 19:04:03.309 - 5: DEBUG: attempt 1: server 127.0.0.1:9001: connected
    app_test.go:1073: 19:04:03.312 - 5: DEBUG: new connection from 127.0.0.1:33438
    app_test.go:1073: 19:04:03.312 - 4: DEBUG: new connection from 127.0.0.1:38442
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x12f8e42]

goroutine 58 [running]:
testing.tRunner.func1(0xc0003fc000)
	/usr/lib/go-1.13/src/testing/testing.go:874 +0x41a
panic(0x154fb00, 0x1949de0)
	/usr/lib/go-1.13/src/runtime/panic.go:679 +0x1d4
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.handleErr(0xc00011d9c8)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/yaml.go:294 +0xb2
panic(0x154fb00, 0x1949de0)
	/usr/lib/go-1.13/src/runtime/panic.go:679 +0x1d4
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.yaml_parser_parse_node(0xc000166000, 0xc0001662b0, 0x100000000000000, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/parserc.go:564 +0x4c2
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.yaml_parser_state_machine(0xc000166000, 0xc0001662b0, 0xc0000ce280)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/parserc.go:170 +0x1c6
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.yaml_parser_parse(0xc000166000, 0xc0001662b0, 0x3ff83337460)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/parserc.go:129 +0x8a
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.(*parser).peek(0xc000166000, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/decode.go:106 +0x42
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.(*parser).parse(0xc000166000, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/decode.go:149 +0x38
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.(*parser).parseChild(0xc000166000, 0xc0000ce280, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/decode.go:197 +0x2a
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.(*parser).document(0xc000166000, 0x300000000000030)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/decode.go:206 +0xa8
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.(*parser).parse(0xc000166000, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/decode.go:159 +0x154
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.unmarshal(0xc00022e480, 0x38, 0x238, 0x1512620, 0xc000196ca0, 0x0, 0x0, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/yaml.go:161 +0x270
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.Unmarshal(...)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/yaml.go:89
github.com/canonical/go-dqlite/app.fileUnmarshal(0xc00046aa80, 0x1e, 0x15cf6b8, 0x9, 0x1512620, 0xc000196ca0, 0x0, 0x127ed30)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/app/files.go:71 +0x174
github.com/canonical/go-dqlite/app.New(0xc00046aa80, 0x1e, 0xc000196c80, 0x3, 0x4, 0xc000196c80, 0x0, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/app/app.go:113 +0x2218
github.com/canonical/go-dqlite/app_test.newAppWithDir(0xc0003fc000, 0xc00046aa80, 0x1e, 0xc00011df58, 0x1, 0x1, 0x0, 0xc000481d00)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/app/app_test.go:1079 +0x190
github.com/canonical/go-dqlite/app_test.TestNew_JoinerRestart(0xc0003fc000)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/app/app_test.go:79 +0x2da
testing.tRunner(0xc0003fc000, 0x15edc28)
	/usr/lib/go-1.13/src/testing/testing.go:909 +0xd6
created by testing.(*T).Run
	/usr/lib/go-1.13/src/testing/testing.go:960 +0x36c
FAIL	github.com/canonical/go-dqlite/app	0.698s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants