VictoriaLogs: Victorialogs: Errors when writing via JSON stream API after version 1.1.0

Is your question request related to a specific component?

Victorialogs

Describe the question in detail

I encountered an issue when writing logs using the JSON stream API. Here are the details:

Environment:

Version: Starting from v1.1.0 Previous version (v1.0.0): Worked perfectly without issues

Problem:

Starting from version 1.1.0, writing logs via the JSON stream API results in the following errors:

2025-01-07T08:04:47.139Z        info    VictoriaMetrics/lib/logstorage/log_rows.go:160  ignoring too long log entry with the estimated size 3247862 bytes, since it exceeds the limit 2097152
2025-01-07T08:04:47.148Z        error   VictoriaMetrics/app/vlinsert/jsonline/jsonline.go:58    jsonline: cannot read line #16740 in /jsonline request: remoteAddr="11.151.222.165:37454", requestURI="/insert/jsonline?_stream_fields=stream&_time_field=date&_msg_field=log.message": cannot read the next line: unexpected EOF

Questions:

Is this issue caused by the logs being truncated when their size exceeds a certain limit? Are there any recommended solutions or configurations to avoid these errors?

Troubleshooting docs

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 19 (11 by maintainers)

Commits related to this issue

Most upvoted comments

Are there solutions to prioritize write stability (even if queries become slower)? Can I adjust parameters to isolate write and query resource usage?

AFAIK there’s no such plan yet. @func25 could you help evaluate whether this would be a good addition?

My writing batches data to a certain size before sending it. A 5MB batch and a 10MB batch are essentially the same in terms of data; the 10MB batch is just a combination of two 5MB batches. Below is the main code for my writing. Could you please help me check if there are any issues?

type VictoriaLogsReq struct {
	Log    *VMLog `json:"log"`
	Date   string `json:"date"`
	Stream string `json:"stream"`
}

func encodeReq(rows []*VMLog) ([]byte, int) {
	var failCnt int
	var buffer bytes.Buffer
	for _, row := range rows {
		date := row.Time.Format(time.RFC3339Nano)
		vlogEntry := VictoriaLogsReq{
			Log:    row,
			Date:   date,
			Stream: row.Target,
		}
		vlogData, err := json.Marshal(vlogEntry)
		if err != nil {
			log.Errorf("error marshaling VictoriaLogsEntry: %v", err)
			failCnt++
			continue
		}

		// 将每个 JSON 对象写入缓冲区,并以换行符分隔
		buffer.Write(vlogData)
		buffer.WriteString("\n")
	}
	return buffer.Bytes(), failCnt
}

func httpPost(client *http.Client, addr string, body []byte) ([]byte, error) {
	req, err := http.NewRequest("POST", addr, bytes.NewReader(body))
	if err != nil {
		return nil, err
	}
	req.Header.Set("Content-Type", "application/stream+json")
	resp, err := client.Do(req)
	if err != nil || resp == nil {
		return nil, fmt.Errorf("error post %w", err)
	}

	defer func() {
		_ = resp.Body.Close()
	}()
	rspBody, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		return nil, fmt.Errorf("error get body from response %w", err)
	}
	if resp.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("unexpected response code=%d", resp.StatusCode)
	}

	return rspBody, nil
}