syslog-ng-users August 2011 archive
Main Archive Page > Month Archives  > syslog-ng-users archives
syslog-ng-users: Re: [syslog-ng] Store syslog occurrence frequen

Re: [syslog-ng] Store syslog occurrence frequency instead of adding all of them to the DB

From: Martin Holste <mcholste_at_nospam>
Date: Thu Aug 18 2011 - 20:10:25 GMT
To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu>

I'd add on to this by using a crc function to hash the message and
store that in a column to make the unique check very fast:
CREATE TABLE mylogs (
  id BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT,
  timestamp TIMESTAMP NOT NULL,
  crc INT UNSIGNED NOT NULL,
  count INT UNSIGNED NOT NULL DEFAULT 0,
  msg TEXT,
  UNIQUE KEY (crc)
);
INSERT INTO mylogs (timestamp, crc, msg) values($timestamp, CRC32($msg),
$msg) ON DUPLICATE KEY UPDATE count=count+1, timestamp=$timestamp;

On Thu, Aug 18, 2011 at 1:33 PM, system@ra-schaal.de
<system@ra-schaal.de> wrote:
> Am 18.08.2011 19:52, schrieb Marcos Tang:
>
>> Can those records being "processed" some how and when I search the MySQL
>> DB, I only see ONE record list the following only?
>>
>> Total occurrenceMessage content
>> =====================
>> 14,400File system is full
>>
>
> canīt you use something like
>
> INSERT INTO tables VALUES (date,logstring) ON duplicate KEY UPDATE date;
>
> ?
>
> florian
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq