Skip to content
Advertisement

Read the log files and get the entries between two dates

I want to extract some information from the access log file that matches a keyword and between two dates. For ex. I want to find log entries between two dates that contains text “passwd”. For now, I am using the following command but not getting the correct results:

fgrep "passwd" * | awk '$4 >= "[20/Aug/2017" && $4 <= "[22/Aug/2017"'

Date format is [22/Feb/2017:17:28:42 +0000].

I have searched and look at this post too extract data from log file in specified range of time but not exactly understand how to use it.

Edits:

Following are the example entries of the access log files,
xxx-access_log:xx.xx.xx.xx - - [22/Feb/2017:17:30:02 +0000] "GET /cms/usr/extensions/get_tree.inc.php?GLOBALS[root_path]=/etc/passwd%00 HTTP/1.1" 404 39798
xxx-access_log:xx.xx.xx.xx - - [22/Feb/2017:17:31:12 +0000] "GET /cgi-bin/libs/smarty_ajax/index.php?_=&f=update_intro&page=../../../../../../../../../../../../../../../../../../etc/passwd%00 HTTP/1.1" 404 30083
xxx-access_log:xx.xx.xx.xx - - [22/Feb/2017:17:31:19 +0000] "GET /download/libs/smarty_ajax/index.php?_=&f=update_intro&page=../../../../../../../../../../../../../../../../../../etc/passwd%00 HTTP/1.1" 404 27982
xxx-access_log:xx.xx.xx.xx - - [22/Feb/2017:17:31:24 +0000] "GET /sites/libs/smarty_ajax/index.php?_=&f=update_intro&page=../../../../../../../../../../../../../../../../../../etc/passwd%00 HTTP/1.1" 404 35256
xxx-access_log:xx.xx.xx.xx - - [22/Feb/2017:17:28:32 +0000] "GET /modx/manager/media/browser/mcpuk/connectors/php/Commands/Thumbnail.php?base_path=/etc/passwd%00 HTTP/1.1" 404 6956
xxx-access_log:xx.xx.xx.xx - - [22/Feb/2017:17:28:42 +0000] "GET /modx/manager/media/browser/mcpuk/connectors/php/Commands/Thumbnail.php?base_path=/etc/passwd%00 HTTP/1.1" 404 6956

Thanks for help in advance!

Advertisement

Answer

The link you quoted would be used if you know 2 specific strings that appear in your log file. That command will search for the first string and display all lines until it finds the second string and then stops.

In your case, if you want generic date manipulation, you might be better off with perl and one of the date/time modules. Most (if not all) of those have built-in date comparison routines, and many of them will take the date in almost any format imaginable … and the ones that don’t typically provide the ability to specify the date format.

(If you’re just using dates and not using times, then Date::EzDate is my favorite, and probably the easiest to learn and implement quickly.)

Shell commands are probably not going to do a good job of date manipulation.

User contributions licensed under: CC BY-SA
4 People found this is helpful
Advertisement