Skip to content

Finally got around to set up my branch properly. #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sdelmas wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Finally got around to set up my branch properly. #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ff97488
1) Adding Cassandra log file parser.
sdelmas Aug 7, 2013
8048706
Adding some test data and a yaml to use for testing (basically the st…
sdelmas Sep 24, 2013
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions bin/gl_tail
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@ elsif !File.exist?(file)
exit
end

$LOAD_PATH.unshift(File.dirname(__FILE__) + '/../lib')

require File.dirname(__FILE__) + '/../lib/gl_tail.rb'

if defined? @print
Expand Down
206 changes: 206 additions & 0 deletions cassandra_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,206 @@
# Only use spaces to indent, not tabs

servers:
server1:
source: local
files: ./test/system1.log
parser: cassandra
color: 0.2, 1.0, 0.2, 1.0
server2:
source: local
files: ./test/system2.log
parser: cassandra
color: 0.2, 1.0, 0.4, 1.0
server3:
source: local
files: ./test/system3.log
parser: cassandra
color: 0.2, 1.0, 0.6, 1.0
server_errors:
source: local
files: ./test/system4.log
parser: cassandra
color: 0.2, 1.0, 0.8, 1.0
config:
dimensions: 1200x600
min_blob_size: 0.004
max_blob_size: 0.04
highlight_color: orange
bounce: false
left_column:
size: 25
alignment: -0.99
blocks:
info:
order: 0
size: 10
auto_clean: false
show: total
hosts:
order: 1
size: 3
sites:
order: 2
size: 10
content:
order: 3
size: 5
show: total
color: 1.0, 0.8, 0.4, 1.0
status:
order: 4
size: 10
color: 1.0, 0.8, 0.4, 1.0
types:
order: 5
size: 5
color: 1.0, 0.4, 0.2, 1.0
users:
order: 6
size: 10
smtp:
order: 7
size: 5
logins:
order: 8
size: 5
database:
order: 9
size: 10
firewall:
order: 10
size: 15
action:
order: 11
size: 15
auto_clean: false
show: total
int:
order: 11
size: 5
rule:
order: 11
size: 5
sourceinterface:
order: 12
size: 15
sourcehost:
order: 13
size: 15
sourceport:
order: 14
size: 15
mappers:
order: 15
size: 8
errors:
order: 16
size: 10
show: total
color: 1.0, 0.0, 0.0, 0.0
dropped messages:
order: 17
size: 5
show: total
color: 1.0, 0.0, 0.0, 0.0
warnings:
order: 18
size: 10
show: total
color: 1.0, 1.0, 0.0, 0.0
compacted:
order: 19
size: 5
show: total
color: 1.0, 1.0, 0.2, 0.0
flushing:
order: 20
size: 5
show: total
color: 1.0, 1.0, 0.2, 0.0
sqlselect:
color: 1.0, 0.8, 0.4, 1.0
sqlupdate:
color: 1.0, 0.2, 0.2, 1.0
sqlinsert:
color: 1.0, 0.5, 0.5, 0.0

right_column:
size: 25
alignment: 0.99
blocks:
urls:
order: 0
size: 15
slow requests:
order: 1
size: 5
show: average
referrers:
order: 2
size: 10
user agents:
order: 3
size: 5
color: 1.0, 1.0, 1.0, 1.0
mail from:
order: 4
size: 5
mail to:
order: 5
size: 5
viruses:
order: 6
size: 5
rejections:
order: 7
size: 5
color: 1.0, 0.2, 0.2, 1.0
warnings:
order: 8
size: 5
destinationinterface:
order: 9
size: 15
ipprotocol:
order: 10
size: 15
sourcedestination:
order: 11
size: 15
destinationhost:
order: 12
size: 15
destinationport:
order: 13
size: 15
agents:
order: 14
size: 8
show: total
auto_clean: false
color: 0.5, 0.5, 0.5, 0.5
dse servers:
order: 15
size: 20
auto_clean: false
show: total
cassandra servers:
order: 16
size: 20
auto_clean: false
show: total
loads:
order: 17
size: 20
auto_clean: false
show: total
schemas:
order: 18
size: 20
auto_clean: false
show: total

resolver:
reverse_ip_lookups: true
reverse_timeout: 0.5
61 changes: 50 additions & 11 deletions dist/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,15 +93,39 @@ config:
order: 14
size: 15
mappers:
order: 11
order: 15
size: 8
errors:
order: 16
size: 10
show: total
color: 1.0, 0.0, 0.0, 0.0
dropped messages:
order: 17
size: 5
show: total
color: 1.0, 0.0, 0.0, 0.0
warnings:
order: 18
size: 10
show: total
color: 1.0, 1.0, 0.0, 0.0
compacted:
order: 19
size: 5
show: total
color: 1.0, 1.0, 0.2, 0.0
flushing:
order: 20
size: 5
show: total
color: 1.0, 1.0, 0.2, 0.0
sqlselect:
color: 1.0, 0.8, 0.4, 1.0
color: 1.0, 0.8, 0.4, 1.0
sqlupdate:
color: 1.0, 0.2, 0.2, 1.0
color: 1.0, 0.2, 0.2, 1.0
sqlinsert:
color: 1.0, 0.5, 0.5, 0.0

color: 1.0, 0.5, 0.5, 0.0

right_column:
size: 25
Expand Down Expand Up @@ -147,22 +171,37 @@ config:
order: 11
size: 15
destinationhost:
order: 11
size: 15
destinationport:
order: 12
size: 15
info:
destinationport:
order: 13
size: 15
auto_clean: false
show: total
agents:
order: 14
size: 8
show: total
auto_clean: false
color: 0.5, 0.5, 0.5, 0.5
dse servers:
order: 15
size: 20
auto_clean: false
show: total
cassandra servers:
order: 16
size: 20
auto_clean: false
show: total
loads:
order: 17
size: 20
auto_clean: false
show: total
schemas:
order: 18
size: 20
auto_clean: false
show: total

resolver:
reverse_ip_lookups: true
Expand Down
53 changes: 53 additions & 0 deletions lib/gl_tail/parsers/cassandra.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# cassandra.rb - OpenGL visualization of your server traffic
# Copyright 2013 Sven Delmas <sven@datastax.com>
#
# Licensed under the GNU General Public License v2 (see LICENSE)
#

# Parser which handles cassandra/system.log (standard log setup) from Apache Cassandra
class CassandraParser < Parser
def parse( line )
# main line parse
_, priority, thread, date, time, fileName, lineNumber, message = /(\S+) \[(\S+)\] (\S+) (\S+) (\S+) \(line (\S+)\) (.*)/.match(line).to_a

if message
# We got a message, so let's parse it for interesting stuff
_, cassandra_version = /Cassandra version: (\S+)/.match(message).to_a
_, dse_version = /DSE version: (\S+)/.match(message).to_a
_, dropped_messages = /(\S+) messages dropped in last/.match(message).to_a
_, compacted = /Compacted (\S+) sstables to/.match(message).to_a
_, load_endpoint, load = /Endpoint (\S+) state changed LOAD = (\S+)/.match(message).to_a
_, schema_endpoint, schema = /Endpoint (\S+) state changed SCHEMA = (\S+)/.match(message).to_a
_, flushing = /Completed flushing (\S+)/.match(message).to_a

if dropped_messages
add_activity(:block => 'dropped messages', :name => server.name)
end
if compacted
add_activity(:block => 'compaction done', :name => server.name)
end
if flushing
add_activity(:block => 'flushing', :name => server.name)
end

# Events to pop up
if cassandra_version
server_name = server.name + '=' + cassandra_version
server_message = server.name + ' started version: ' + cassandra_version
add_event(:block => 'cassandra servers', :name => server_name, :message => server_message, :update_stats => true, :color => [0.0, 1.0, 0.0, 0.0])
end
if dse_version
server_name = server.name + '=' + dse_version
server_message = server.name + ' started version: ' + dse_version
add_event(:block => 'dse servers', :name => server_name, :message => server_message, :update_stats => true, :color => [0.0, 1.0, 0.0, 0.0])
end
if schema_endpoint
schema_name = schema_endpoint
schema_message = schema_endpoint + ' changed schema: ' + schema
add_activity(:block => 'schemas', :name => schema_name)
end
add_event(:block => 'errors', :name => server.name, :message => message, :update_stats => true, :color => [1.0, 0.0, 0.0, 0.0]) if priority == "ERROR"
add_event(:block => 'warnings', :name => server.name, :message => message, :update_stats => true, :color => [1.0, 1.0, 0.0, 0.0]) if priority == "WARN"
end
end
end
Loading