HTTP request result truncated #893

New issue

2023-09-21T09:56:03Z

dlozeve commented

2023-09-21 09:56:03 +00:00

(Migrated from github.com)

(request-text (http-get "https://www.wikipedia.org")) does not contain the full HTML contents of the page, it is truncated after 30050 characters.

This makes request-json crash when the response JSON object gets truncated and becomes invalid:

*** ERROR IN std/text/json/input#read-json-string/reader -- *** ERROR IN read-json [IOError]: Incomplete JSON object; EOF reached
--- irritants: #<BufferedStringReader #29> 
--- continuation backtrace:
0  raise                                                                              
1  std/text/json/input#read-json-string/reader                                                                                                                           (std/text/json/env#raise-invalid-token _input215841_ _char215860_)
2  std/text/json/input#read-json-hash/reader                                                                                                                             (std/text/json/input#read-json-object/reader _input216057_ _env216058_)
3  std/text/json/input#read-json-hash/reader                                                                                                                             (std/text/json/input#read-json-object/reader _input216057_ _env216058_)
4  std/text/json/input#read-json-list-next/reader                                                                                                                        (std/text/json/input#read-json-object/reader _input215879_ _env215880_)
5  std/text/json/input#read-json-list/reader                                                                                                                             (std/text/json/input#read-json-list-next/reader _input215953_ _env215954_)
6  std/text/json/input#read-json-hash/reader                                                                                                                             (std/text/json/input#read-json-object/reader _input216057_ _env216058_)
7  std/text/json/input#read-json-hash/reader                                                                                                                             (std/text/json/input#read-json-object/reader _input216057_ _env216058_)
8  std/text/json/input#read-json-list-next/reader                                                                                                                        (std/text/json/input#read-json-object/reader _input215879_ _env215880_)
9  std/text/json/input#read-json-list/reader                                                                                                                             (std/text/json/input#read-json-list-next/reader _input215953_ _env215954_)
10 std/text/json/input#read-json-hash/reader                                                                                                                             (std/text/json/input#read-json-object/reader _input216057_ _env216058_)
11 std/text/json/util#string->json-object                                                                                                                                (std/text/json/input#read-json-object/reader _reader224353_ (let ((__obj340180 (mak...
12 (interaction)                                                                      (stdin)@27:1                                                                       (std/net/request#request-json req)

`(request-text (http-get "https://www.wikipedia.org"))` does not contain the full HTML contents of the page, it is truncated after 30050 characters. This makes `request-json` crash when the response JSON object gets truncated and becomes invalid: ``` *** ERROR IN std/text/json/input#read-json-string/reader -- *** ERROR IN read-json [IOError]: Incomplete JSON object; EOF reached --- irritants: #<BufferedStringReader #29> --- continuation backtrace: 0 raise 1 std/text/json/input#read-json-string/reader (std/text/json/env#raise-invalid-token _input215841_ _char215860_) 2 std/text/json/input#read-json-hash/reader (std/text/json/input#read-json-object/reader _input216057_ _env216058_) 3 std/text/json/input#read-json-hash/reader (std/text/json/input#read-json-object/reader _input216057_ _env216058_) 4 std/text/json/input#read-json-list-next/reader (std/text/json/input#read-json-object/reader _input215879_ _env215880_) 5 std/text/json/input#read-json-list/reader (std/text/json/input#read-json-list-next/reader _input215953_ _env215954_) 6 std/text/json/input#read-json-hash/reader (std/text/json/input#read-json-object/reader _input216057_ _env216058_) 7 std/text/json/input#read-json-hash/reader (std/text/json/input#read-json-object/reader _input216057_ _env216058_) 8 std/text/json/input#read-json-list-next/reader (std/text/json/input#read-json-object/reader _input215879_ _env215880_) 9 std/text/json/input#read-json-list/reader (std/text/json/input#read-json-list-next/reader _input215953_ _env215954_) 10 std/text/json/input#read-json-hash/reader (std/text/json/input#read-json-object/reader _input216057_ _env216058_) 11 std/text/json/util#string->json-object (std/text/json/input#read-json-object/reader _reader224353_ (let ((__obj340180 (mak... 12 (interaction) (stdin)@27:1 (std/net/request#request-json req) ```

Sign in to join this conversation.