"Data read, but end of buffer not reached"

My packed file has reached 1.4gb and I now get this error immediately upon attempting to read it:

Uncaught Error: Data read, but end of buffer not reached
    at checkedRead (/Users/beau/p/zed-run/node_modules/msgpackr/dist/node.cjs:130:10)
    at unpack (/Users/beau/p/zed-run/node_modules/msgpackr/dist/node.cjs:77:10)
    at exports.readPacked (/Users/beau/p/zed-run/utilities.js:223:32)

My readPacked function is simply:

unpack(fs.readFileSync(path));

Issue Analytics

State:
Created 2 years ago
Comments:15 (4 by maintainers)

Top GitHub Comments

2reactions

LekoArtscommented, Feb 2, 2022

Hi @kriszyp 👋

We’re running into this same error with Gatsby now. It seems that the recent 1.5.3 release caused this.

The output one gets is:

"internal-data-bridge" threw an error while running the sourceNodes lifecycle:

Data read, but end of buffer not reached

File: node_modules/gatsby/src/internal-plugins/internal-data-bridge/gatsby-node.js:55:5

  Error: Data read, but end of buffer not reached
  
  - node.cjs:178 checkedRead
    [browser-ssr-tsx]/[msgpackr]/dist/node.cjs:178:10
  
  - node.cjs:91 Packr.unpack
    [browser-ssr-tsx]/[msgpackr]/dist/node.cjs:91:10
  
  - node.cjs:150 Packr.decode
    [browser-ssr-tsx]/[msgpackr]/dist/node.cjs:150:15
  
  - index.js:440 Object.get
    [browser-ssr-tsx]/[lmdb-store]/index.js:440:63
  
  - caching.js:37 Object.get
    [browser-ssr-tsx]/[lmdb-store]/caching.js:37:17
  
  - lmdb-datastore.ts:148 Object.getNode
    [browser-ssr-tsx]/[gatsby]/src/datastore/lmdb/lmdb-datastore.ts:148:16
  
  - index.ts:18 getNode
    [browser-ssr-tsx]/[gatsby]/src/datastore/index.ts:18:18
  
  - public.js:767 createNode
    [browser-ssr-tsx]/[gatsby]/src/redux/actions/public.js:767:19
  
  - public.js:863 
    [browser-ssr-tsx]/[gatsby]/src/redux/actions/public.js:863:21
  
  - index.js:23 
    [browser-ssr-tsx]/[redux-thunk]/lib/index.js:23:18
  
  - redux.js:554 boundActionCreator
    [browser-ssr-tsx]/[redux]/lib/redux.js:554:12
  
  - api-runner-node.js:79 createNode
    [browser-ssr-tsx]/[gatsby]/src/utils/api-runner-node.js:79:20
  
  - gatsby-node.js:55 forEach
    [browser-ssr-tsx]/[gatsby]/src/internal-plugins/internal-data-bridge/gatsby-node.js:55:5
  
  - Array.forEach
  
  - gatsby-node.js:53 Object.exports.sourceNodes
    [browser-ssr-tsx]/[gatsby]/src/internal-plugins/internal-data-bridge/gatsby-node.js:53:20
  
  - api-runner-node.js:462 runAPI
    [browser-ssr-tsx]/[gatsby]/src/utils/api-runner-node.js:462:22

internal-data-bridge is something from us so that’s why it is printing that.

You can reproduce this by running npm init gatsby -y issue-34, let the installation run and then do two npm run build in a row. On the second run you’ll see this error. If you do a npm run clean in between (it removes the LMDB store data) in between, it works.

Let me know if you need any further information. Thanks a lot!

1reaction

LekoArtscommented, Feb 2, 2022

Works with 1.5.4 again. Thanks again!