It uses the windows FindFirstChangeNotification which means it does use very little resources. (A nonsense benchmark, Idle time 4886h, ChkMagick 6s)
Every [Section] in the ini file gets its own thread. Within a section files are handled one at a time,
the files waiting in the From= folder is the queue.
get ChkMagick
ChkMagick.zip
chkmagick.ini for your needs and...
chkmagick /install
chkmagick.ini you need to restart the service, one way isnet stop chkmagicknet start chkmagick
chkmagick /uninstall
log\ folder next to chkmagick.exe the logs are written there, otherwise next to the exe.
A new set of files is started every month, yymm.
| File | Content |
|---|---|
| ChkMagick-yymm-log.txt | moved files, executed commands, times, depending on LogL=
|
| ChkMagick-yymm-error.txt | errors, timeouts, duplicates, missing folders |
| ChkMagick-yymm-service.txt | start / stop and the configuration read for every section |
| ChkMagick-yymm-profile.txt | file date (LogL>2) and the full exif/iptc profile (LogL>3) for each file, use it to find the names of metadata %parameters |
| ChkMagick-section_n.tst | output from Tstn=
|
| ChkMagick-section.ini | list of processed files (duplicate test), saved on stop, read on start |
From= folder (and subfolders if SubT=1) wakes the section.
Every 15s it also checks again after an earlier i/o error.
MaxProc=: if too many processes are running the section waits here, the file stays in the queue.
Sleep= ms is waited (not the first round after start).
!p123! etc are read and removed.
SecretKey=: the time hash is tested, invalid files are moved to SecretDest= or deleted.
Extn=/ToDn= or To=, filename by File=.
DuplDest= and run MvApp1..3,
unless Dupl=1.
Copn, Appn (if Iffn and size limits ok), Pushn, Tstn.
Drop=1 deletes the moved file.
MdToApp= runs if a new destination folder was created.
chkmagick-computername.ini it is prefered
chkmagick.ini
chkmagick.ini does not exists, the service stops
# ; . rem [-- [== ,
so [== IMPORT ==] can be used as a header. Text after ] on a section line is ignored.
_ or start the next line with space or tab to continue it.
= are removed, keys are not case sensitive.
[Setup] and [DateTime] is a folder to listen to.
If the From= folder does not exist the section is skipped (a note in the error log).
LogL= or LogLevel=. If both are given the long name wins.
| [Setup] | Description | Default | Example |
|---|---|---|---|
| Root= | where the commands reside when Appn= has no path | Root=c:\programs\imagemagick | |
| Prio= | process priority class Low|Normal|High|Realtime
(also Idle|-1|0|1, låg|hög) | Low=Idle | |
| Time= | max time to wait for each command, ms (min 1000) | 18000 | |
| Kill= | 0|1, terminate the process if not finished within Time | 0 | |
| Sleep= | wait before each file is processed, ms | 1000 | |
| LogL= LogLevel= | log level 0..9, 0=one line per command, 1=normal, 3=config+profile, 5..9=debug | 1 | LogL=2 |
| File= FileNameMask= | default filename mask in the To folder | File=%srcfn_simple | |
| FileID= FilenameID= | default map for %filenameid | ||
| MinFileName= MinFileNameLength= | shorter filenames are extended with _yymmdd_random | 12 | MinFileNameLength=0 |
| MaxFileName= MaxFileNameLength= | longer filenames are cut | 60 | |
| FileNameYearCheck= | how many years back a date in the filename is trusted, see filedates | 10 | |
| MinFileSize= | min filesize to run commands, file is still moved. 16384, 100KB, 10MB | -1 = no check | |
| MaxFileSize= | max filesize to run commands | -1 = no check | |
| SubT= SubTree= | 0|1, also listen to subfolders of From | 0 | |
| App= Cmd= Dir= | default Appn/Cmdn/Dirn when not given in a command | ||
| SecK= SecretKey= | secret key for the time hash test | SecretKey | |
| SecT= SecretTime= | seconds per time hash window, 0 = no test | 0 | |
| MaxProc=new | see MaxProc | 0 = no limit | |
| MaxProcName= | |||
| MaxProcWait= | 1000 | ||
| MaxProcTimeout= | 0 |
Note, Dupl=, DuplDest= and LogRotate= are not read from [Setup], set them in each section.
| [Section] | Description | Default | Example |
|---|---|---|---|
| Folders | |||
| From= | the folder to listen to, when a file is copied/created here the process begins. MUST be an existing folder. | From=c:\upload\img | |
| SubT= | 0|1, also listen to subfolders | [Setup]SubT | |
| To= | the folder to move the file to, created if it does not exist.
%parameters can be used, ~ is replaced by \ | To=c:\arkiv\%yyyy\%mm | |
| Extn= | list of extensions, n=1..16, first match decides the folder | Ext1=.7z;.zip;.rar | |
| ToDn= ToDirn= | move to this folder if extension in Extn, otherwise To= | ToD1=c:\upload\zip | |
| MdToApp= | command to run when a new destination folder was created | MdToApp=c:\cmd\ftpdirs.exe | |
| MdToCmd= | parameters to MdToApp, the new folder is added last | ||
| MdToSto= | standard output (append) for MdToApp | ||
| Filenames | |||
| File= FileNameMask= | filename mask in the To folder, the extension is kept | [Setup]File | File=%srcfn_simple%_U%now12 |
| FileID= FilenameID= | map for %filenameid, name=id;name2=id2.
The first name found in the filename gives the id.
If no match, %filenameid=0 and the rest of the commands are skipped | FileID=kvitto=1;faktura=2 | |
| MinFileNameLength= MaxFileNameLength= | see [Setup] | [Setup] | |
| FileNameYearCheck= | see [Setup] | [Setup] | |
| Duplicates | |||
| Dupl= | 0|1. With 0 a file already processed (name, project id, size and Crc32, list cleared every night)
is moved to DuplDest and the commands are not run. With 1 the commands are run anyway. If an identical file already exists in To the new file is deleted (when DuplDest is empty) and the commands are run on the existing file. | 0 | |
| DuplDest= | folder for duplicates | chkmagick\duplicates\ | DuplDest=c:\dupl |
| Comp= | compare with this folder instead of To (with Dupl=1).
If a file with the same name but not the same content exists the filename gets _n. Use it when files move in stages and the name must be unique in the final folder. | ||
| MvAppn= | n=1..3, commands to run when a file is moved to DuplDest.
Same keys as below with prefix Mv, MvCmdn, MvDirn, MvStin ... | ||
| Processing | |||
| Time= | max time for each command, ms | [Setup]Time | Time=200000 |
| Kill= | 0|1, terminate the process when Time is reached. With 0 the process keeps running in the background and the next file starts | [Setup]Kill | |
| Sleep= | wait before each file, ms | [Setup]Sleep | |
| Prio= | default priority for the commands | [Setup]Prio | |
| Root= | path for commands without path | [Setup]Root | |
| MinFileSize= MaxFileSize= | size limits to run commands | [Setup] | |
| Exif= | 0|1, set the filedate from the exif date of the moved file | 0 | |
| Drop= | 0|1, delete the file after all commands | 0 | |
| MaxProc - limit running processes new | |||
| MaxProc= | max number of running processes with the same exe name (all processes on the computer, not only from this section). If reached, wait until one has finished. The files wait in the From folder. | 0 = no limit | MaxProc=6 |
| MaxProcName= | exe name(s) to count, separated by ; or ,. Used for all commands in the section. | exe of Appn | MaxProcName=mariadb.exe |
| MaxProcWait= | how often to count again while waiting, ms (min 100) | 1000 | MaxProcWait=2500 |
| MaxProcTimeout= | max wait per file, ms. Then the command starts anyway (note in the error log) | 0 = wait forever | MaxProcTimeout=0 |
| Security, time hash | |||
| SecretKey= | if set together with SecretTime, the first 1KB of the file must contain a valid time hash (current or previous time window) | [Setup]SecretKey | |
| SecretTime= | seconds per time window | [Setup]SecretTime | SecretTime=1800 |
| SecretDest= InvalidDest= | folder for files without a valid hash, if empty the file is deleted | ||
| Logging | |||
| LogL= LogLevel= | log level for this section | [Setup]LogL | |
| LogR= LogRotate= | rotate the log file when larger than, min 8KB. 16384, 100KB, 8MB | no rotation | LogRotate=8MB |
| Commands, n=1..16, run in order for every file | |||
| Appn= | the command to run, without path Root= is used | [Section]App | App1=convert.exe |
| Cmdn= | parameters to the command, with %parameters | [Section]Cmd | Cmd1=%f -thumbnail 120 %fil.jpg |
| Dirn= | the folder to run the command in, created if it does not exist | path of Appn | |
| Prin= Prion= | priority for this command | [Section]Prio | |
| Iffn= | run only if the original filename matches this regular expression | Iff1=(?i)^Kvitto_|^Receipt_ | |
| Stin= StandardInputn= | file for standard input. If it does not exist (after ~1s) the command is not run | Sti1=%f | |
| Ston= StandardOutputn= | file for standard output, appended | Sto1=c:\tmp\%fnseu4.text | |
| Copn= Copyn= | copy a file before Appn is run, source destination separated by space, ; or ,.
An existing destination is not overwritten | Cop1=c:\dir\file.txt c:\dir\%yyyy\%fil.txt | |
| Exin= Exifn= | read metadata (%px, %py, exif ...) from this file instead of the moved file | Exif8=c:\arkiv\s\%fil_ext1 | |
| Pusn= Pushn= | add text to the push string, read later with %pop | Push7='%fno','%fil' | |
| Puan= PushAn= | same for %popa | PushA8=,%px,%py | |
| Pubn= PushBn= | same for %popb | ||
| Tstn= Testn= | write the replaced text to ChkMagick-section_n.tst, to test %parameters | Tst1=%fil %yyyy %px | |
A key Appn with only Cmdn uses App= from the section,
that way one program can be called many times:
App=convert.exe + Cmd1=... Cmd2=....
tcc.exe and cmd.exe are started in a new console.
Some programs lose the first parameter, then use Cmdn=0 param1 param2.
%fdat etc. if the server locale is wrong.CurrencyString CurrencyFormat NegCurrFormat ThousandSeparator DecimalSeparator CurrencyDecimals
DateSeparator ShortDateFormat LongDateFormat TimeSeparator TimeAMString TimePMString ShortTimeFormat LongTimeFormat
ShortMonthNames1..12 LongMonthNames1..12 ShortDayNames1..7 LongDayNames1..7
| Prefix | Sets | Read with |
|---|---|---|
| !p123! | project id | %projid |
| !s123! | sub project id | %delpid |
| !o123! | organisation / part id | %partid |
| !u123! | user id | %userid |
| !mtext! | metadata, can be repeated | %metadata, %meta |
| !dtext! | directory tag (xdtext), can be repeated | %directories, %meta |
In To, File, Comp, Cmdn, Dirn, Stin, Ston, Copn, Exifn, Pushn, Tstn, SecretDest
you can use the following parameters.
Names are lowercase letters, digits and _.
|
A picture file taken 2005-apr-08: img_0123.jpg is dropped 2014-dec-24 14:45:15 in c:\home\web\upload\img\project_23\ The section below handles the file for the "result" below [Nav-img] From=c:\home\web\upload\img SubT=1 To=c:\home\nav\web\upload\img File=%yymmdd_%fnr_%r%fc | |
| %par | Description | Result |
|---|---|---|
| Source (original) file | ||
| %src %fo | original file with path | c:\home\web\upload\img\project_23\img_0123.jpg |
| %srcfne %fno | original filename with extension | img_0123.jpg |
| %srcfn %fn | original filename without extension | img_0123 |
| %srcext | original extension, lowercase | .jpg |
| %srcpath | original path | c:\home\web\upload\img\project_23\ |
| %src_unix %srcfn_unix %srcpath_unix | same with / | c:/home/web/upload/img/project_23/ |
| %srcfn_simple %fns, %fnh | simple filename, only a..z 0..9 _ - ! ( ), åäö ⇒ aao.
With SubT=1 the last subfolder is added as prefix folder_ | img_0123 |
| %srcfne_simple %fnse, %fnhe | same with extension | img_0123.jpg |
| %src_simple | simple filename, no folder prefix | img_0123 |
| %fnseu2..%fnseu8 | simple filename + _ + unique string + extension | img_0123_x8y4.jpg |
| %yymmddfns | file date + simple filename, old dates in the name are removed | 050408_img_0123 |
| %src1 | filename before the first ! $ @ | |
| %src2 | second part of the filename between ! | |
| %srcfn_nr %fnr | filenumber of the filename, max 16 | img0123 |
| %srcfn_nr_uc %fnru | same, uppercase | IMG0123 |
| %fnr8, %fnr12, %fnr24, %fnr32, %fnr48 | filenumber max 8..48 chars, %fnrunn uppercasealso %srcfn_nrnn, %srcfn_nrnn_uc | |
| %srchash2..6 %fh2..%fh6 | small hash from filename + filesize, 2..6 chars | a3 |
| %srcdate %fdat | file date and time | 2005-04-08 12:30:00 |
| Source folder parts (c:\home\web\upload\img\project_23\) | ||
| %srcpath2 %fd | the folder of the file | project_23 |
| %srcpath3 %fdb | parent folder | img |
| %srcpath4 | parent of parent | upload |
| %srcpath_id %fdd | last part of the folder after _ | 23 |
| %srcprojid | first folder in the path with only digits | |
| Destination (moved) file | ||
| %dst %f | moved file with path | c:\home\nav\web\upload\img\050408_img0123_600.jpg |
| %dst0 %f0 | same, for .gif and video [0] is added = first frame for ImageMagick | ...\anim.gif[0] |
| %dstfne %fme | filename with extension | 050408_img0123_600.jpg |
| %dstfn %fm | filename without extension | 050408_img0123_600 |
| %dstext %fe | extension, lowercase | .jpg |
| %dstpath %fp | path | c:\home\nav\web\upload\img\ |
| %dst_unix | file with / | c:/home/nav/web/upload/img/050408_img0123_600.jpg |
| %dstfn_unix %fu | file with /, no extension | c:/home/nav/web/upload/img/050408_img0123_600 |
| %dstpath_unix %fpu | path with / | c:/home/nav/web/upload/img/ |
| %fil | the target filename, without path or extension | 050408_img0123_600 |
| %file | target filename + extension | 050408_img0123_600.jpg |
| %ext1 | smart extension: .png for gif/bmp/ico/png/tif, else .jpg | .jpg |
| %ext2 | same, but large png/tif (>1920x1080) gives .jpg | .jpg |
| %fil_ext1 %fil_ext2 | %fil + %ext1 / %ext2 | 050408_img0123_600.jpg |
| %dst2 %dst2fn, %dst2fne, %dst2ext, %dst2path %dst2_unix, %dst2fn_unix, %dst2path_unix | destination without !p123! prefixes | |
| Ids and archive | ||
| %projid, %delpid, %partid, %userid | ids from prefixes or folder | 23 |
| %metadata, %directories, %meta | text from !m..! and !d..!, %meta = both | |
| %arkiv %arkivdos | archive folder: proj\projid (projid>7), part\partid (partid>7), user\userid, super\1 (id=1) or pub\0 | proj\23 |
| %arkivunix | same with / | proj/23 |
| %filenameid %fnid | id from FileID= map, 0 skips the rest of the commands | 1 |
| Image and metadata (from the moved file or Exifn=) | ||
| %width %px | pixel width, null if unknown | 1024 |
| %height %py | pixel height, null if unknown | 768 |
| %rot %srcimrotate | ImageMagick rotation from exif orientation | -rotate 90 |
| %r %srcexifrotate | exif orientation 0,6(=90),8(=-90) | 6 |
| %latitude %lat | GPS latitude, negative for S | 59.3293 |
| %longitude %lng | GPS longitude, negative for W | 18.0686 |
| %magicktitle | title from exif/iptc/jpeg comment | |
| %magickcam | camera, exposure, f-number, ISO, focal length | Canon EOS 5D | 1/125s | f/8 | ISO100 | 50mm |
| %magickcopy | artist, author, copyright | |
| %magicktags | keywords and categories | |
| %magickindex %magickindex255 | all words above for a search index, 255 = max 255 chars | |
| %resnnn | resolution (dpi) to get nnn pixels on the longest side, for .pdf/.eps/.ps | -r%res600 |
| %tagname | any exif/iptc/eps value in -profile.txt, written lowercase.
Quotes and \ are escaped for sql | %gpsimagedirection %creationdate |
| File date (see filedates) | ||
| %yyyy | year, 4 digits | 2005 |
| %yy | year, 2 digits | 05 |
| %mm | month 01-12 | 04 |
| %dd | day 01-31 | 08 |
| %yymmdd | date, 6 digits | 050408 |
| %hh | hour 00-23 | 12 |
| %nn | minute 00-59 | 30 |
| %ss | second 00-59 | 00 |
| Current date and time | ||
| %year2 | yy | 14 |
| %year4 | yyyy | 2014 |
| %date4 | yymm | 1412 |
| %date6 | yymmdd | 141224 |
| %date8 | yyyymmdd | 20141224 |
| %date10 | yyyy-mm-dd | 2014-12-24 |
| %time2 | hh | 14 |
| %time4 | hhmm | 1445 |
| %time6 | hhmmss | 144515 |
| %time8 | hh:mm:ss | 14:45:15 |
| %now8 | yymmddhh | 14122414 |
| %now10 | yymmddhhmm | 1412241445 |
| %now11 | yymmdd-hhmm | 141224-1445 |
| %now12 | yymmddhhmmss | 141224144515 |
| %now13 | yymmdd-hhmmss | 141224-144515 |
| %now16 | yyyy-mm-dd_hh-mm | 2014-12-24_14-45 |
| %now19 | yyyy-mm-dd_hh-mm-ss | 2014-12-24_14-45-15 |
| Unique, random and counters | ||
| %unique2..%unique8 | unique string, the same for all commands of one file | x8y4 |
| %uniqueint %uniquei | unique number, the same for all commands of one file | 1804289383 |
| %random2 %random3 %random4 | new random string every time, 00..zz / 000..zzz / 0000..zzzz | x8y |
| %counter %fc | counter per section 00..ZZ | 00 |
| Push and pop | ||
| %pop | all Pushn= texts for this file so far | |
| %popclear | same, and clear it | |
| %popa %popb | PushAn= / PushBn= texts | |
| Other | ||
| %content %fi | content of the file on one line, max 32KB | |
| %usr %user, %username | windows user of the service | SYSTEM |
%yyyy, %mm ... is taken from, in order:
FileNameYearCheck= years.
[Setup] Root=C:\Programs\ImageMagick Time=12000 LogL=2 [Nav-img] From=c:\home\web\upload\img To=c:\home\web\pic\o\%yyyy\%mm File=%yymmddfns%_%fh4 # thumbnail 120x120 App1=convert.exe Cmd1=%f0 -auto-orient -thumbnail "30000@" -gravity center -crop 120x120+0+0 +repage _ -background white -flatten -quality 88 %fil.jpg Dir1=c:\home\web\pic\t\%yyyy\%mm # small screen App2=convert.exe Cmd2=%f0 -auto-orient -resize 600x700 -strip -quality 80 %fil_ext1 Dir2=c:\home\web\pic\s\%yyyy\%mm Push3='%fno','%fil','%yyyy-%mm-%dd %hh:%nn:%ss','%latitude','%longitude','%magicktitle' Exif4=c:\home\web\pic\s\%yyyy\%mm\%fil_ext1 PushA4=,%px,%py App5=C:\Program Files\MariaDB 11.8\bin\mariadb.exe Dir5=C:\Program Files\MariaDB 11.8\bin Cmd5=-D web -u chk -ppw -e "CALL ImgsImport(%pop%popa);"
[sql-queue] SubT=1 From=c:\ftp\queue To=c:\ftp\proj\%srcpath2\done File=%srcfn_simple%_U%now12 Dupl=1 Time=200000 Kill=1 MaxProc=6 MaxProcName=mariadb.exe MaxProcWait=2500 MaxProcTimeout=0 App1=C:\Programs\ConvMPKL\ConvMPKL.exe Cmd1=0 -u=1 -p=%srcpath2 -t="%fno" %f %fil.sql Dir1=c:\ftp\proj\%srcpath2\done App2=C:\Program Files\MariaDB 11.8\bin\mariadb.exe Dir2=c:\ftp\proj\%srcpath2\done Cmd2=-D ndh -u chk -ppw --default-character-set=latin1 -e "source %fil.sql"
[mail-pickup] From=c:\chkmagick\pickup To=c:\chkmagick\done File=%date6%_%time4%_%random4%_%fn App1=c:\programs\command\msmtp.exe Cmd1=-t Sti1=%f
More info/questions from: Subject: ChkMagick