nodejs-logging-winston: Library still throws if we try to log a large object

Environment details

  • OS: Ubuntu 18.04
  • Node.js version: v10.18.0
  • npm version: 6.13.4
  • @google-cloud/logging-winston version: 3.0.2

Steps to reproduce

  1. Suppose veryLargeString is a variable containing a string with about 100,000 characters.
  2. Try to run the “Quickstart” code snippet in the README (having appropriately set the authentication environment variables), but try to run the following line: logger.info(veryLargeString);. The code will run without issues, and log the string to Stackdriver Logging.
  3. Now try to run the following line: logger.info('foo', { veryLargeString });. The code throws with the following error trace:
(node:20708) UnhandledPromiseRejectionWarning: Error: 3 INVALID_ARGUMENT: Log entry with size 1.00M exceeds maximum size of 256.0K
    at Object.callErrorFromStatus (/tmp/logging.D0fP/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Http2CallStream.call.on (/tmp/logging.D0fP/node_modules/@grpc/grpc-js/build/src/client.js:96:33)
    at Http2CallStream.emit (events.js:203:15)
    at process.nextTick (/tmp/logging.D0fP/node_modules/@grpc/grpc-js/build/src/call-stream.js:98:22)
    at process._tickCallback (internal/process/next_tick.js:61:11)
(node:20708) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:20708) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Am I to believe that this is not intended behavior, as #404 should fix this?

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 3
  • Comments: 16 (5 by maintainers)

Most upvoted comments

Forgive my directness, but what’s the point of closing an issue on the basis of not yet having a viable solution to it, only to have to open a new issue later, isn’t the entire purpose of logging the issue to track the identification and resolution?

Leaving the issue open to retain visibility of the problem seems like a reasonable approach in such a situation, no?

A while ago, I did a fix in base nodejs-logging library to support more fields for truncation - here is a PR 1177. Thats said, I never exposed this logic in nodejs-logging-winston - please let me know if this is something you would like me to do. In addition, recently I changed the logic to include partialSuccess to be true by default - this at least make sure that only oversized entries will be dropped from request (if of course the size of the request itself does not goes beyond limit as specified in log-limits) - in this case while the error still would be returned, in response it will indicate which entries were dropped. Also, I believe we should implement log splitting as mentioned in LogSplit - this is a feature we can work on if there will be enough demand

In my case, to solve the issue I created a wrapper for the LoggingWinston with a custom truncate logic before send the message to the logger.

const TransportStream = require('winston-transport');
const { LoggingWinston } = require('@google-cloud/logging-winston')

class wrapperGoogleLogsTransport extends TransportStream {
  constructor(opts) {
    super(opts)
    this.googleLogsTransport = new LoggingWinston(opts)
  }

  log(info, callback) {
   // here you can write your custom logic, like truncate or maybe chunk info in parts and send by part
    const sizeInfo = JSON.stringify(info).length
    if (sizeInfo > 256000) {
      info = JSON.stringify(info).substring(0, 256000)
    }
    this.googleLogsTransport.log(info, callback)
  }
}

export default wrapperGoogleLogsTransport

We encounter the same issue on our Cloud Run instances, and fixed it by adding the following fields to LoggingWinston transport options. docs

import { LoggingWinston } from '@google-cloud/logging-winston';


const transport = new LoggingWinston({
  useMessageField: false,
  redirectToStdout: true,
});

This way the error is not throwing and the log object appears as a truncated string.

To maintain the log info as json we implemented @benitogonzalezh’s solution (thank you!) with few modifications.

import { LoggingWinston } from '@google-cloud/logging-winston';
import {
  isArray,
  isObject,
  isString,
  truncate,
  cloneDeep,
} from 'lodash';

export function truncateObjectDeep(object: any, length: number): any {
  const copyObj = cloneDeep(object);
  const truncateDeep = (obj: any): any => {
    if (isString(obj) && obj.length > length) {
      return `${truncate(obj, { length })} [truncated]`;
    }
    if (isArray(obj)) {
      return obj.map(i => truncateDeep(i));
    }
    if (isObject(obj)) {
      Object.keys(obj).forEach(key => {
        (obj as any)[key] = truncateDeep((obj as any)[key]);
      });
      return obj;
    }

    return obj;
  };
  return truncateDeep(copyObj);
}


class GoogleLogsTransport extends LoggingWinston {
  log(info: any, callback: (err: Error | null, apiResponse?: {}) => void) {
    const sizeInfo = JSON.stringify(info).length;
    if (sizeInfo > 250000) {
      info = truncateObjectDeep(info, 5000); // the max length for string values
      info.truncated = true;
    }
    super.log(info, callback);
  }
}


const transport = new GoogleLogsTransport({
    useMessageField: false,
    redirectToStdout: true,
  });


We assume that most large size logs will have few fields that are significantly larger. This will not work if our log is a very large object but all values are small in size. length variable should be adjusted to support your use case.

I’d use that @losalex !

I’m going to close this for now because we don’t have a viable way to solve this. The cost of dynamically truncating arbitrarily nested strings inside of custom objects is too much and too complex to not be surprising.

it would be nice if this never threw. it’s not a good user experience. instead we should perhaps log the error message or a broken json object truncated like a string.

we should open another issue with our decision here.

Could you propose an acceptable algorithm that might work the way we’ed need on objects? I’m not sure if there is a generic enough computationally viable solution to this.

like data could be preserved based on how close it is to the top of the json object. doing a shallow first count of bytes. Then arbitrary keys at a certain depth would be truncated after bytes are exhausted. It’s hard to say this is what every user will expect/