跳到正文
Hacker News-首页· olexsmir·· 5 小时前精选AI 评分84

Go 语言发布全新的 JSON API:encoding/json/v2 与 jsontext 正式稳定

A new Go API for JSON

AI 导读

Go 语言团队宣布全新的 `encoding/json/v2` 和 `encoding/json/jsontext` 软件包在 Go 1.27 中稳定并普遍可用。新版本底层采用基于语法和语义分离的设计,支持流式处理、一等公民选项配置及更严格的 RFC 8259 规范校验,同时在底层实现了与旧版 v1 的双向兼容。

推荐理由

新发布的 encoding/json/v2 和 jsontext 解决了旧版 API 在语法规范、流式处理和性能上的局限,并实现了与 v1 的双向兼容,为 Go 语言生态提供了更安全高效的 JSON 处理方案。

正文 · 原文

Michael Pratt, Joe Tsai, Daniel Martí, Johan Brandhorst-Satzkorn, Roger Peppe, Chris Hines, and Damien Neil
4 November 2026

Introduction

JavaScript Object Notation (JSON) is a simple data interchange format, the most popular data format used on the internet, and widely read and written by Go programs. Go has included support for JSON in the standard library since well before Go 1.0. We first we wrote about support for JSON in Go more than 15 years ago!

Over time, packages evolve with the needs of their users, and encoding/json is no exception. Last year, we wrote about a new experimental API for JSON designed to bring long-awaited improvements and fixes.

After additional feedback and refinement, the new encoding/json/v2 and encoding/json/jsontext packages are stable and generally available in Go 1.27! This post is an updated version of the article introducing the experiment, describing the problems with encoding/json, and an introduction to the new packages.

Problems with encoding/json

Overall, encoding/json has held up well. The idea of marshaling and unmarshaling arbitrary Go types with some default representation in JSON, combined with the ability to customize the representation, has proven to be highly flexible. However, in the years since its introduction, various users have identified numerous shortcomings.

Behavior flaws

There are various behavioral flaws in encoding/json:

  • Imprecise handling of JSON syntax: Over the years, JSON has seen increased standardization in order for programs to properly communicate. Generally, decoders have become stricter at rejecting ambiguous inputs, to reduce the chance that two implementations will have different (successful) interpretations of a particular JSON value.

    • encoding/json currently accepts invalid UTF-8, whereas the latest Internet Standard (RFC 8259) requires valid UTF-8. The default behavior should report an error in the presence of invalid UTF-8, instead of introducing silent data corruption, which may cause problems downstream.

    • encoding/json currently accepts objects with duplicate member names. RFC 8259 does not specify how to handle duplicate names, so an implementation is free to choose an arbitrary value, merge the values, discard the values, or report an error. The presence of a duplicate name results in a JSON value without a universally agreed upon meaning. This could be exploited by attackers in security applications and has been exploited before (as in CVE-2017-12635). The default behavior should err on the side of safety and reject duplicate names.

  • Leaking nilness of slices and maps: JSON is often used to communicate with programs using JSON implementations that do not allow null to be unmarshaled into a data type expected to be a JSON array or object. Since encoding/json marshals a nil slice or map as a JSON null, this may lead to errors when unmarshaling by other implementations. A survey indicated that most Go users prefer that nil slices and maps are marshaled as an empty JSON array or object by default.

  • Case-insensitive unmarshaling: When unmarshaling, a JSON object member name is resolved to a Go struct field name using a case-insensitive match. This is a surprising default, a potential security vulnerability, and a performance limitation.

  • Inconsistent calling of methods: Due to an implementation detail, MarshalJSON methods declared on a pointer receiver are inconsistently called by encoding/json. While regarded as a bug, this cannot be fixed as too many applications depend on the current behavior.

API deficiencies

The API of encoding/json can be tricky or restrictive:

  • It is difficult to correctly unmarshal from an io.Reader. Users often write json.NewDecoder(r).Decode(v), which fails to reject trailing junk at the end of the input.

  • Options can be set on the Encoder and Decoder types, but cannot be used with the Marshal and Unmarshal functions. Similarly, types implementing the Marshaler and Unmarshaler interfaces cannot make use of the options and there is no way to plumb options down the call stack. For example, the Decoder.DisallowUnknownFields option loses its effect when calling a custom UnmarshalJSON method.

  • The Compact, Indent, and HTMLEscape functions write to a bytes.Buffer instead of something more flexible like a []byte or io.Writer. This limits the usability of those functions.

Performance limitations

Setting aside internal implementation details, the public API commits it to certain performance limitations:

  • MarshalJSON: The MarshalJSON interface method forces the implementation to allocate the returned []byte. Also, the semantics require that encoding/json verify that the result is valid JSON and also to reformat it to match the specified indentation.

  • UnmarshalJSON: The UnmarshalJSON interface method requires that a complete JSON value be provided (without any trailing data). This forces encoding/json to parse the JSON value to be unmarshaled in its entirety to determine where it ends before it can call UnmarshalJSON. Afterwards, the UnmarshalJSON method itself must parse the provided JSON value again.

  • Lack of streaming: Even though the Encoder and Decoder types operate on an io.Writer or io.Reader, they buffer the entire JSON value in memory. The Decoder.Token method for reading individual tokens is allocation-heavy and there is no corresponding API for writing tokens.

Furthermore, if the implementation of a MarshalJSON or UnmarshalJSON method recursively calls the Marshal or Unmarshal function, then the performance becomes quadratic.

Trying to fix encoding/json directly

Introducing a new, incompatible major version of a package is a heavy consideration. If possible, we should try to fix the existing package.

While it is relatively easy to add new features, it is difficult to change existing features. Unfortunately, these problems are inherent consequences of the existing API, making them practically impossible to fix within the Go 1 compatibility promise.

We could in principle declare separate names, such as MarshalV2 or UnmarshalV2, but that is tantamount to creating a parallel namespace within the same package. This leads us to encoding/json/v2 (henceforth called v2), where we can make these changes within a separate v2 namespace in contrast to encoding/json (henceforth called v1).

Building on encoding/json/jsontext

Before discussing the v2 API, we first introduce the encoding/json/jsontext package that lays the foundation for improvements to JSON in Go.

JSON serialization in Go can be broken down into two primary components:

  • syntactic functionality that is concerned with processing JSON based on its grammar, and
  • semantic functionality that defines the relationship between JSON values and Go values.

We use the terms “encode” and “decode” to describe syntactic functionality and the terms “marshal” and “unmarshal” to describe semantic functionality. We aim to provide a clear distinction between functionality that is purely concerned with encoding versus that of marshaling.

This diagram provides an overview of this separation. Purple blocks represent types, while blue blocks represent functions or methods. The direction of the arrows approximately represents the flow of data. The bottom half of the diagram, implemented by the jsontext package, contains functionality that is only concerned with syntax, while the upper half, implemented by the json/v2 package, contains functionality that assigns semantic meaning to syntactic data handled by the bottom half.

The basic API of jsontext is the following:

package jsontext

type Encoder struct { ... }
func NewEncoder(io.Writer, ...Options) *Encoder
func (*Encoder) WriteValue(Value) error
func (*Encoder) WriteToken(Token) error

type Decoder struct { ... }
func NewDecoder(io.Reader, ...Options) *Decoder
func (*Decoder) ReadValue() (Value, error)
func (*Decoder) ReadToken() (Token, error)

type Kind byte
type Value []byte
func (Value) Kind() Kind
type Token struct { ... }
func (Token) Kind() Kind

The jsontext package provides functionality for interacting with JSON at a syntactic level and derives its name from RFC 8259, section 2 where the grammar for JSON data is literally called JSON-text. Since it only interacts with JSON at a syntactic level, it does not depend on Go reflection.

The Encoder and Decoder provide support for encoding and decoding JSON values and tokens. The constructors accept variadic options that affect the particular behavior of encoding and decoding. Unlike the Encoder and Decoder types declared in v1, the new types in jsontext avoid muddling the distinction between syntax and semantics and operate in a truly streaming manner.

A JSON value is a complete unit of data and is represented in Go as a named []byte. It is identical to RawMessage in v1, which is now a type alias to jsontext.Value. A JSON value is syntactically composed of one or more JSON tokens. A JSON token is represented in Go as the opaque Token type with constructors and accessor methods. It is analogous to Token in v1 but is designed represent arbitrary JSON tokens without allocation.

To resolve the fundamental performance problems with the MarshalJSON and UnmarshalJSON interface methods, we need an efficient way of encoding and decoding JSON as a streaming sequence of tokens and values. In v2, we introduce the MarshalJSONTo and UnmarshalJSONFrom interface methods that operate on an Encoder or Decoder, allowing the methods’ implementations to process JSON in a purely streaming manner. Thus, the json package need not be responsible for validating or formatting a JSON value returned by MarshalJSON, nor would it need to be responsible for determining the boundaries of a JSON value provided to UnmarshalJSON. These responsibilities belong to the Encoder and Decoder.

Introducing encoding/json/v2

Building on the jsontext package, we now introduce the encoding/json/v2 package. It is designed to fix the problems in v1, while remaining familiar to users of the v1 package. Our goal is that usages of v1 will operate mostly the same if directly migrated to v2.

Here, we primarily cover the high-level API of v2. For examples on how to use it, we encourage readers to study the examples in the v2 package, the “Working with JSON” tutorial, or read Anton Zhiyanov’s blog covering the topic.

The basic API of v2 is the following:

package json

func Marshal(in any, opts ...Options) (out []byte, err error)
func MarshalWrite(out io.Writer, in any, opts ...Options) error
func MarshalEncode(out *jsontext.Encoder, in any, opts ...Options) error

func Unmarshal(in []byte, out any, opts ...Options) error
func UnmarshalRead(in io.Reader, out any, opts ...Options) error
func UnmarshalDecode(in *jsontext.Decoder, out any, opts ...Options) error

The Marshal and Unmarshal functions have a signature similar to v1, but accept options to configure their behavior. The MarshalWrite and UnmarshalRead functions directly operate on an io.Writer or io.Reader, avoiding the need to temporarily construct an Encoder or Decoder just to write or read from such types. The MarshalEncode and UnmarshalDecode functions operate on a jsontext.Encoder and jsontext.Decoder and is actually the underlying implementation of the previously mentioned functions. Unlike v1, options are a first-class argument to each of the marshal and unmarshal functions, greatly extending the flexibility and configurability of v2. There are several options available in v2 which are not covered by this article.

Type-specified customization

Similar to v1, v2 allows types to define their own JSON representation by satisfying particular interfaces.

type Marshaler interface {
    MarshalJSON() ([]byte, error)
}
type MarshalerTo interface {
    MarshalJSONTo(*jsontext.Encoder) error
}

type Unmarshaler interface {
    UnmarshalJSON([]byte) error
}
type UnmarshalerFrom interface {
    UnmarshalJSONFrom(*jsontext.Decoder) error
}

The Marshaler and Unmarshaler interfaces are identical to those in v1. The new MarshalerTo and UnmarshalerFrom interfaces allow a type to represent itself as JSON using a jsontext.Encoder or jsontext.Decoder. This allows options to be forwarded down the call stack, since options can be retrieved via the Options accessor method on the Encoder or Decoder.

See the OrderedObject example for how to implement a custom type that maintains the ordering of JSON object members.

Caller-specified customization

In v2, the caller of Marshal and Unmarshal can also specify a custom JSON representation for any arbitrary type, where caller-specified functions take precedence over type-defined methods or the default representation for a particular type.

func WithMarshalers(*Marshalers) Options

type Marshalers struct { ... }
func MarshalFunc[T any](fn func(T) ([]byte, error)) *Marshalers
func MarshalToFunc[T any](fn func(*jsontext.Encoder, T) error) *Marshalers

func WithUnmarshalers(*Unmarshalers) Options

type Unmarshalers struct { ... }
func UnmarshalFunc[T any](fn func([]byte, T) error) *Unmarshalers
func UnmarshalFromFunc[T any](fn func(*jsontext.Decoder, T) error) *Unmarshalers

MarshalFunc and MarshalToFunc construct a custom marshaler that can be passed to a Marshal call using WithMarshalers to override the marshaling of particular types. Similarly, UnmarshalFunc and UnmarshalFromFunc support similar customization for Unmarshal.

The ProtoJSON example demonstrates how this feature allows serialization of all proto.Message types to be handled by the protojson package.

Behavior differences

While v2 aims to behave mostly the same as v1, its behavior has changed in some ways to address problems in v1, most notably:

  • v2 reports an error in the presence of invalid UTF-8.

  • v2 reports an error if a JSON object contains a duplicate name.

  • v2 marshals a nil Go slice or Go map as an empty JSON array or JSON object, respectively.

  • v2 unmarshals a JSON object into a Go struct using a case-sensitive match from the JSON member name to the Go field name.

  • v2 redefines the omitempty tag option to omit a field if it would have encoded as an “empty” JSON value (which are null, "", [], and {}).

  • v2 reports an error when trying to serialize a time.Duration, which currently has no default representation, but provides options to let the caller decide.

For most behavior changes, there is a struct tag option or caller-specified option that can configure the behavior to operate under v1 or v2 semantics, or even other caller-determined behavior. See “Migrating to v2” for more information.

Performance optimizations

The Marshal performance of v2 is roughly at parity with v1. Sometimes it is slightly faster, but other times it is slightly slower. The Unmarshal performance of v2 is significantly faster than v1, with benchmarks demonstrating improvements of up to 10x.

In order to obtain greater performance gains, existing implementations of Marshaler and Unmarshaler should migrate to also implement MarshalerTo and UnmarshalerFrom, so that they can benefit from processing JSON in a purely streaming manner. For example, recursive parsing of OpenAPI specifications in UnmarshalJSON methods significantly hurt performance in a particular service of Kubernetes (see kubernetes/kube-openapi#315), while switching to UnmarshalJSONFrom improved performance by orders of magnitude.

For more benchmark details, see the go-json-experiment/jsonbench repository.

On top of structural improvements, we’re continuing to invest in performance optimizations in the implementation for cases where the v2 implementation is slower than v1. You can expect additional improvements in Go 1.28.

Compatibility

The Go 1 compatibility promise is a core guarantee from the Go project that allows developers to build applications that will stand the test of time and continue working on future versions of Go.

The original encoding/json and new encoding/json/v2 packages are of course both subject to the Go 1 compatibility promise. Applications using v1 will continue to be supported indefinitely and need not ever migrate to v2.

That said, when designing the v2 packages we wanted to do better than keeping v1 supported. We wanted to ensure that v1 and v2 are compatible with one another so that packages using different versions can work together. Doing so allows a smooth ecosystem migration rather than forcing users to use one version or another due to the choices of their dependencies.

For instance, suppose your package uses v2 to marshal a struct containing a field with a type defined in another package. That package may use the original v1 MarshalJSON method to define its marshaling behavior. No problem, v2 supports MarshalJSON methods, so marshaling will continue working as expected.

Vice versa, suppose your package uses v1 to marshal the same struct. The other package may use the v2 MarshalJSONTo method for better performance. No problem here either, as the v1 implementation supports v2’s MarshalJSONTo method under the hood.

We’ve achieved this by designing the new packages such that, under the hood, v1 is implemented in terms of v2. As mentioned above, v2 supports a variety of Options. Each behavior difference is controlled by one of the these options. Thus, v1 can be implemented in terms of v2 by passing the full set of v1 compatibility options.

With this approach, backwards-compatible additions such as new methods (e.g., MarshalJSONTo) or new struct tags (e.g., embed) are automatically supported to the benefit of existing v1 users.

Migration

Implementing v1 in terms of v2 options has the additional benefit of allowing gradual migration of callsites.

encoding/json documents the full set of behavior differences between v1 and v2, and which option controls that behaviors.

Many applications can migrate directly to v2 with no visible behavior changes, but applications that depend on v1 behavior can use the options to change only the behaviors they depend on.

For example, we can select all v2 behavior except for allowing duplicate names:

json.Marshal(..., json.DefaultOptionsV2(), jsontext.AllowDuplicateName(true))

Alternatively, we can select all v1 behavior except for disallowing duplicate names:

json.Marshal(..., jsonv1.DefaultOptionsV1(), jsontext.AllowDuplicateName(false))

The encoding/json/v2 Migration Guide goes into more detail, covering a variety of scenarios for approaching migration of an application from v1 to v2.

More information

For additional references on working with the new JSON packages, see:

A long time coming

The planning for a new major version of encoding/json spanned years. In late 2020, spurred on by the inability to fix issues in the current package, Daniel Martí (one of the maintainers of encoding/json) first drafted his thoughts on what a hypothetical v2 package should look like. Separately, after previous work on the Go API for Protocol Buffers, Joe Tsai was disappointed that the protojson package needed to use a custom JSON implementation because encoding/json was neither capable of adhering to the stricter JSON standard that the Protocol Buffer specification required, nor of efficiently serializing JSON in a streaming manner.

Believing a brighter future for JSON was both beneficial and achievable, Daniel and Joe joined forces to brainstorm on v2 and started to build a prototype (with the initial code being a polished version of the JSON serialization logic from the Go protobuf module). Over time, a few others (Roger Peppe, Chris Hines, Johan Brandhorst-Satzkorn, Damien Neil, Austin Clements, and Michael Pratt) joined the effort by providing design review, code review, and regression testing. Many of the early discussions are publicly available in our recorded meetings and meeting notes. Later discussions are available in our encoding/json/v2 working group meeting notes.

This work has been public since the beginning, and we increasingly involved the wider Go community, first with a GopherCon talk and discussion posted in late 2023, formal proposal posted in early 2025, and finally adopting encoding/json/v2 as a Go experiment in Go 1.25 for wider-scale testing by all Go users.

The v2 effort took about 6 years to reach this point, incorporating feedback from many contributors and also gaining valuable empirical experience from use in production settings.

It’s worth noting that it’s largely been developed and promoted by people not employed by Google, demonstrating that the Go project is a collaborative endeavor with a thriving global community dedicated to improving the Go ecosystem.

Thank you to everyone that contributed!

来源:Hacker News-首页 · go.dev