Capturing unused fields while decoding a JSON object with circe

  • A+

Suppose I have a case class like the following, and I want to decode a JSON object into it, with all of the fields that haven't been used ending up in a special member for the leftovers:

import io.circe.Json  case class Foo(a: Int, b: String, leftovers: Json) 

What's the best way to do this in Scala with circe?

(Note: I've seen questions like this a few times, so I'm Q-and-A-ing it for posterity.)


There are a couple of ways you could go about this. One fairly straightforward way would be to filter out the keys you've used after decoding:

import io.circe.{ Decoder, Json, JsonObject }  implicit val decodeFoo: Decoder[Foo] =   Decoder.forProduct2[Int, String, (Int, String)]("a", "b")((_, _)).product(     Decoder[JsonObject]   ).map {     case ((a, b), all) =>       Foo(a, b, Json.fromJsonObject(all.remove("a").remove("b")))   } 

Which works as you'd expect:

scala> val doc = """{ "something": false, "a": 1, "b": "abc", "0": 0 }""" doc: String = { "something": false, "a": 1, "b": "abc", "0": 0 }  scala> io.circe.jawn.decode[Foo](doc) res0: Either[io.circe.Error,Foo] = Right(Foo(1,abc,{   "something" : false,   "0" : 0 })) 

The disadvantage of this approach is that you have to maintain code to remove the keys you've used separately from their use, which can be error-prone. Another approach is to use circe's state-monad-powered decoding tools:

import import cats.instances.either._ import io.circe.{ ACursor, Decoder, Json }  implicit val decodeFoo: Decoder[Foo] = Decoder.fromState(   for {     a <- Decoder.state.decodeField[Int]("a")     b <- Decoder.state.decodeField[String]("b")     rest <- StateT.inspectF((_: ACursor).as[Json])   } yield Foo(a, b, rest) ) 

Which works the same way as the previous decoder (apart from some small differences in the errors you'll get if decoding fails):

scala> io.circe.jawn.decode[Foo](doc) res1: Either[io.circe.Error,Foo] = Right(Foo(1,abc,{   "something" : false,   "0" : 0 })) 

This latter approach doesn't require you to change the used fields in multiple places, and it also has the advantage of looking a little more like any other decoder you'd write manually in circe.


:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: