Have you ever had to retrieve millions of items in a json array that multi gigabyte payload? This might help!
This library reads a json stream, picks out items from a json array and writes them to a channel.
Given the file test.json
{
"list": [
{"key": "value1"},
{"key": "value2"},
{"key": "value3"},
{"key": "value4"}
]
}
Call the library
in := j2c.StartFileReader("test.json")
out := j2c.ReadObjects(in, "list")
for o := range out {
fmt.Println(o)
}
in
is a instance whose struct implements Reader defined as
type Reader interface {
Next() (b byte, eof bool)
}
out := j2c.ReadObjects(in, "list")
is the line that does the work.
The output out
is a channel that receives objects as strings as they are made available.
Output
{"key": "value1"}
{"key": "value2"}
{"key": "value3"}
{"key": "value4"}
Currently, in small tests the library produces over 500k items per second.