Search
As of October, 2016, Embarcadero is offering a free release
of Delphi (Delphi
10.1 Berlin Starter Edition ). There
are a few restrictions, but it is a welcome step toward making
more programmers aware of the joys of Delphi. They do say
"Offer may be withdrawn at any time", so don't delay if you want
to check it out. Please use the
feedback link to let
me know if the link stops working.
Support DFF - Shop
If you shop at Amazon anyway, consider
using this link.
We receive a few cents from each
purchase. Thanks
Support DFF - Donate
If you benefit from the website, in terms of
knowledge, entertainment value, or something otherwise useful,
consider making a donation via PayPal to help defray the
costs. (No PayPal account necessary to donate via credit
card.) Transaction is secure.
Mensa®
Daily Puzzlers
For over 15 years
Mensa Page-A-Day calendars have provided several puzzles a year
for my programming pleasure. Coding "solvers" is most fun,
but many programs also allow user solving, convenient for "fill
in the blanks" type. Below are Amazon links to the
two most recent years.
Mensa®
365 Puzzlers Calendar 2017
Mensa®
365 Puzzlers Calendar 2018
(Hint: If you can
wait, current year calendars are usually on sale in January.)
Contact
Feedback:
Send an
e-mail with your comments about this program (or anything else).
|
| |
Problem Description
A program to read website log files and count
"hits" or other site events based on record content.
Background & Techniques
Our home page has a "hit counter" component which stopped working
sometime in the April to June, 2009 timeframe. It may be fixable, but
in the meantime I wanted to recover the count of home page visits since my
last checkpoint at the end of in March. I save raw log data
routinely to analyze the popularity of program downloads, so I have the data
but needed a program to count home page visit events.
Logs are text files are produced daily which record interactions between
web pages and the outside world including all the events which occur when
user clicks to open a page. The event I want to count is the one
that invokes the FrontPage hit counter program FPCount.exe.
I did try to generalize the program to make easier to apply to solving
other problems.
| The program allows the user to select the folder to be searched for
log files.
Within that folder, a "mask" controls which specific files are
scanned. The rules for file masks are summarized in the
sidebar at right. Typically we see masks like "*.*" (any file name
containing a dot ), or as in our default mask in this case "ex*.log"
since I name my log files as exyymmdd.log where
yymmdd is the year, month and day numbers of the day it contains.
Finally, the specific text to be searched within each record is
a user input. I only need one field in this case, but included
provisions for a second match filed required which is
ignored if left blank.
In this version, only hit count totals and the highest date found
are kept, but future version might include option to save the
specific matching record for further processing. The
date and count totals are kept in a text file named :HitCount.ini. |
File mask rules
A file mask is a string of characters against which file
names may be tested. Here's a summary of the
file masking rules:
- Characters in the string other than * and
? must match
the corresponding position in the file name.
- A ? matches any character in that position.
- An * matches any number of consecutive characters until
the character following the * in the mask is found or the
end of the string is reached.
- Matches are case insensitive.
|
To test the operation, I have included in the download zips a few hundred
records from two late June log files in.
Programmer's Notes
A new feature, at least for me, was using ShBrowseForFolder Windows API
function which displays an Explorer type dialog allowing user to select the
target folder. The GetFolder function contained in a new
USelectFolder unit was adapted from Angus Johnson's
TZip component which I downloaded several years ago from the excellent
Torry.net website but
never got around to implementing.
Once the program has the folder to scan, that folder name and file mask
field is passed to SelectFiles procedure to fill a String list named
FileList with the names of the files to scan. The files are
successively read in a loop, checking each record for matches against the
match fields supplied by the user. When a match is found, the
HitCount integer is incremented and the date extracted from the log
record. Windows logs use a somewhat unusual format for date-time
stamps, at least not one handled automatically by Delphi's date routines,
e,g, 28/Jun/2009:00:15:42. I ended up extracting
the three character month abbreviations, converting them to a month number
and then using then EncodeDate function to convert year, month, and
day numbers to a TDate field, Filedate.
SaveDate and SaveCount fields are saved in HitCounter.ini
file and have no built-in backup/undo features, I have tried to get user
verification before updating the fields; not perfect, but
better than nothing. .
Running/Exploring the Program
Suggestions for Further Explorations
 |
Future uses might require saving the
file names or the records which matched for further analysis. |
|
|
It would be good to save a history of
results so that errors could be corrected, perhaps saving a separate
section in the Ini file which incorporates run date and time in the
section name. By default, the section with the most current
run data would provide the starting values for the next run.
|
|
|
The program should work for non-log
text files except for the date extraction, but i have not tested it.
|
|
Original Date: June 16, 2009 |
Modified:
May 15, 2018 |
|
|
