Skip to content

feat: use bufio scanner to decode events #45

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hazzik wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat: use bufio scanner to decode events #45

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 6 additions & 8 deletions sse-decoder.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
package sse

import (
"bufio"
"bytes"
"io"
)
Expand Down Expand Up @@ -37,19 +38,16 @@ func (d *decoder) dispatchEvent(event Event, data string) {
}

func (d *decoder) decode(r io.Reader) ([]Event, error) {
buf, err := io.ReadAll(r)
if err != nil {
return nil, err
}

var currentEvent Event
dataBuffer := new(bytes.Buffer)
// TODO (and unit tests)
// Lines must be separated by either a U+000D CARRIAGE RETURN U+000A LINE FEED (CRLF) character pair,
// a single U+000A LINE FEED (LF) character,
// or a single U+000D CARRIAGE RETURN (CR) character.
Comment on lines 43 to 46

Copilot AI Nov 30, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The TODO comment on line 44 mentions that SSE lines can be separated by CRLF, LF, or CR, but there are no tests verifying CRLF handling. Since bufio.Scanner handles CRLF differently than the previous bytes.Split approach (it strips the \r), consider adding test cases with CRLF line endings to verify the decoder works correctly with Windows-style line endings.

Example test case:

events, err := Decode(bytes.NewBufferString("data: test\r\nevent: message\r\n\r\n"))

Copilot uses AI. Check for mistakes.
lines := bytes.Split(buf, []byte{'\n'})
for _, line := range lines {
s := bufio.NewScanner(r)

Copilot AI Nov 30, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bufio.Scanner has a default maximum token size of 64KB. If SSE events can contain lines longer than this limit, the scanner will return an error (bufio.ErrTooLong).

If large events are expected, consider using s.Buffer() to increase the maximum token size:

s := bufio.NewScanner(r)
// Optional: increase buffer size if needed
// s.Buffer(make([]byte, 0, 64*1024), maxTokenSize)

Alternatively, if this size limit is acceptable for your use case, consider adding validation or documentation about this constraint.

Suggested change
s := bufio.NewScanner(r)
s := bufio.NewScanner(r)
// Increase buffer size to support large SSE events (default is 64KB)
const maxTokenSize = 1024 * 1024 // 1MB
s.Buffer(make([]byte, 0, 64*1024), maxTokenSize)

Copilot uses AI. Check for mistakes.
for s.Scan() {
line := s.Bytes()

@magnusbaeck magnusbaeck Nov 27, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The buffer size of bufio.Scanner is 64 kB, which might not be enough for some inputs. If anything, I'd suggest using bufio.Reader and its ReadString or ReadBytes methods.

if len(line) == 0 {
// If the line is empty (a blank line). Dispatch the event.
d.dispatchEvent(currentEvent, dataBuffer.String())
Expand Down Expand Up @@ -113,5 +111,5 @@ func (d *decoder) decode(r io.Reader) ([]Event, error) {
// Once the end of the file is reached, the user agent must dispatch the event one final time.
d.dispatchEvent(currentEvent, dataBuffer.String())

return d.events, nil
return d.events, s.Err()
}