Skip to content

Fix harvester_activity_parser regex for Chia 2.7.x log format#400

Open
DFC72 wants to merge 1 commit into
martomi:mainfrom
DFC72:fix-harvester-parser-chia-27x
Open

Fix harvester_activity_parser regex for Chia 2.7.x log format#400
DFC72 wants to merge 1 commit into
martomi:mainfrom
DFC72:fix-harvester-parser-chia-27x

Conversation

@DFC72
Copy link
Copy Markdown

@DFC72 DFC72 commented May 29, 2026

Chia 2.7.x changed the harvester log format in two ways:

  1. Added a 'challenge_hash: ...' prefix before the plot count
  2. Changed 'Found N proofs.' to 'Found N V1 proofs and N V2 qualities.' to support compressed plots

The old regex did not match either change, causing chiadog to never register healthy harvester events and continuously alert as unhealthy despite the farm working correctly.

This fix updates the regex to handle both the old and new log formats, preserving backward compatibility. Tested against Chia 2.7.1 logs.

Chia 2.7.x changed the harvester log format in two ways:

1. Added a 'challenge_hash: <hash> ...' prefix before the plot count
2. Changed 'Found N proofs.' to 'Found N V1 proofs and N V2 qualities.' to support compressed plots

The old regex did not match either change, causing chiadog to never register healthy harvester events and continuously alert as unhealthy despite the farm working correctly.

This fix updates the regex to handle both the old and new log formats, preserving backward compatibility. Tested against Chia 2.7.1 logs.
Copy link
Copy Markdown
Owner

@martomi martomi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this!

Blocking: indentation is corrupt and CI tests are failing
Would be great to also add a test-case for the new format.

Comment on lines +18 to +50
class HarvesterActivityParser:
"""This class can parse info log messages from the chia harvester
You need to have enabled "log_level: INFO" in your chia config.yaml
The chia config.yaml is usually under ~/.chia/mainnet/config/config.yaml
"""
def __init__(self):
logging.debug("Enabled parser for harvester activity - eligible plot events.")
self._regex = re.compile(
r"([0-9T:.+-]*) (?:[-0-9a-zA-Z.]+ )?harvester (?:src|chia).harvester.harvester(?:\s?): INFO\s*"
r"(?:challenge_hash: [0-9a-f]+ \.\.\.)?([0-9]+) plots were "
r"eligible for farming ([0-9a-z.]*)\s*Found ([0-9]+) "
r"(?:V1 )?proofs(?:\.| and [0-9]+ V2 qualities\.) Time: ([0-9.]*) s. "
r"Total ([0-9]*) plots"
)
def parse(self, logs: str) -> List[HarvesterActivityMessage]:
"""Parses all harvester activity messages from a bunch of logs
:param logs: String of logs - can be multi-line
:returns: A list of parsed messages - can be empty
"""
parsed_messages = []
matches = self._regex.findall(logs)
for match in matches:
parsed_messages.append(
HarvesterActivityMessage(
timestamp=dateutil_parser.parse(match[0]),
eligible_plots_count=int(match[1]),
challenge_hash=match[2],
found_proofs_count=int(match[3]),
search_time_seconds=float(match[4]),
total_plots_count=int(match[5]),
)
)
return parsed_messages
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the indentation got corrupted.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry - stupid Cut and paste... try this one?

class HarvesterActivityParser:
"""This class can parse info log messages from the chia harvester
You need to have enabled "log_level: INFO" in your chia config.yaml
The chia config.yaml is usually under ~/.chia/mainnet/config/config.yaml
"""

def __init__(self):
    logging.debug("Enabled parser for harvester activity - eligible plot events.")
    self._regex = re.compile(
        r"([0-9T:.+-]*) (?:[-0-9a-zA-Z.]+ )?harvester (?:src|chia).harvester.harvester(?:\s?): INFO\s*"
        r"(?:challenge_hash: [0-9a-f]+ \.\.\.)?([0-9]+) plots were "
        r"eligible for farming ([0-9a-z.]*)\s*Found ([0-9]+) "
        r"(?:V1 )?proofs(?:\.| and [0-9]+ V2 qualities\.) Time: ([0-9.]*) s. "
        r"Total ([0-9]*) plots"
    )

def parse(self, logs: str) -> List[HarvesterActivityMessage]:
    """Parses all harvester activity messages from a bunch of logs
    :param logs: String of logs - can be multi-line
    :returns: A list of parsed messages - can be empty
    """
    parsed_messages = []
    matches = self._regex.findall(logs)
    for match in matches:
        parsed_messages.append(
            HarvesterActivityMessage(
                timestamp=dateutil_parser.parse(match[0]),
                eligible_plots_count=int(match[1]),
                challenge_hash=match[2],
                found_proofs_count=int(match[3]),
                search_time_seconds=float(match[4]),
                total_plots_count=int(match[5]),
            )
        )
    return parsed_messages

@DFC72 DFC72 requested a review from martomi June 3, 2026 04:20
@DFC72
Copy link
Copy Markdown
Author

DFC72 commented Jun 3, 2026

Is this failing due to Python 2.7?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants