Logging - Dating of logs

andrew bear — Thu, 15 Jan 2026 14:49:46 -0000

The last bit actually should be reordered as follows

Hint: Use menu "Empty->Empty Standby List" to clear the cache, , try your app, File->Refresh to refresh the display, then view what files are cached in the "File Summary" tab which can be ordered by path.

Logging - Dating of logs

andrew bear — Thu, 15 Jan 2026 07:29:58 -0000

Hi Stefan,

Forgot to say my tests did conclusively reveal that it is the combined size of the directory that determines the great majority of the response time rather than the number of files for your function. This makes sense when considering it takes time to copy all these files from disk to the cache.

Also note I have confirmed that your function does cache the files whereas my faster one doesn't using RamMap from SysInternals.

https://learn.microsoft.com/en-gb/sysinternals/downloads/rammap

You could try RamMap with QFileInfo::size() to confirm if it doesn't use cached files.

Hint: Use menu "Empty->Empty Standby List" to clear the cache, File->Refresh to refresh the display, try your app, then view what files are cached in the "File Summary" tab which can be ordered by path.

Cheers
Andy

Logging - Dating of logs

andrew bear — Thu, 15 Jan 2026 07:14:13 -0000

Hi Stefan,

The built in function (which uses QFile::size() ) is fast once the files are all added to the MS Windows file cache but that takes ages for large directories (50 seconds in my case) with it blocking in the meantime and actually slows the rest of operating system down overall who is also fighting for the disk cache.

I've looked at the QT source code (see below) but got lost at the lower levels. QFileInfo.size() is probably much faster as it seems to use fstat (see below), primarily , which uses the MFT (Master File Table) which according to Google AI (see below) is generally cached in memory. Of coarse the only way to find out for sure is to test it.

Cheers
Andy

**QFile::size() **

https://codebrowser.dev/qt5/qtbase/src/corelib/io/qfile.cpp.html

qint64 QFile::size() const
{
return QFileDevice::size(); // for now
}

qint64 QFileDevice::size() const
{
Q_D(const QFileDevice);
if (!d->ensureFlushed())
return 0;
d->cachedSize = d->engine()->size();
return d->cachedSize;
}

**QFileDevice::size() **

qint64 QFileDevice::size() const
{
Q_D(const QFileDevice);
if (!d->ensureFlushed())
return 0;
d->cachedSize = d->engine()->size();
return d->cachedSize;
}

QFileInfo::size()

The following is fast and generally filled by fstat at invocation of QFileInfo for each file.

"{ return d->metaData.size(); }"

The fallback "d->fileEngine->size()" is what QFile ::size() uses and as the code suggests uses cached file data.

https://codebrowser.dev/qt5/qtbase/src/corelib/io/qfileinfo.cpp.html#1394

qint64 QFileInfo::size() const
{
Q_D(const QFileInfo);
return d->checkAttribute<qint64>(
QFileSystemMetaData::SizeAttribute,
d { return d->metaData.size(); },
d {
if (!d->getCachedFlag(QFileInfoPrivate::CachedSize)) {
d->setCachedFlag(QFileInfoPrivate::CachedSize);
d->fileSize = d->fileEngine->size();
}
return d->fileSize;
});
}</qint64>

**From Google AI (which may not be correct as it hashes databases)**

The speed of fstat() on Windows is generally very fast for retrieving file metadata (an O(1) operation),

Speed is for Metadata: fstat() retrieves metadata (like file size, timestamps, permissions) which is typically stored in the file system's Master File Table (MFT) and can be accessed quickly in memory. It is not a measure of data transfer speed.

Overhead Varies: The actual time taken can be a few milliseconds or less per call. The overhead becomes noticeable if a program performs a large number of calls in a tight loop

Logging - Dating of logs

Stefan Zieker — Thu, 15 Jan 2026 06:24:59 -0000

Hi Andy,

I was not aware that the built in function is so slow. I think createProcessAsynchronous with du.exe is the way to go.

Cheers
Stefan

Logging - Dating of logs

andrew bear — Thu, 15 Jan 2026 02:09:23 -0000

Note I nearly have finished a script to create a new timestamped log file each day and also limit the log directory size. However even though the functionality can be implemented in a script it's a rather slow and technical complex way (with a few headaches) for something I think would be worthwhile building in the the app.

My intention is to share this script here because there are parts in it that many will find useful (directory size reading, daily log files, execution of external commands, and a useful exception handler).

The only problems I've had so far with this is getting the directory size in a speedy way.

The inbuilt ScriptCommunicator function scriptFile.getFileSize is very slow on first read as it caches the entire file (it uses QT QFile.size() from QJSEngine which is the Javascript engine used by ScriptCommunicator). For a small 5 file 30kb directory it only takes 3ms. However on a large 319MB directory with 1525 files it takes 50 seconds for the first directory size read and approx 150ms thereafter.

I tried using an external program via scriptThread.createProcessAsynchronous which is much faster for the first directory read at 140-150ms with most of this being overhead of just calling the external program (approx 130ms).

My program is a VBscript so there may be significant program startup time even when not reading directory sizes (directory size readings only took about 10ms as already mentioned judging on the response time when the directory code was commented out). I will eventually try other file types to see if that has significant effect on times. I may try SysInternals du.exe which is said to be superfast for reading directories.

Whilst this directory read timing is fine for me others may need faster response. Any suggestions?

Cheers
Andy

Logging - Dating of logs

andrew bear — Mon, 12 Jan 2026 15:53:07 -0000

Hi all,

For log management purposes it would be useful to have an option create a new log each day and append a date suffix. I note that the current dating option only creates new dated logs on certain conditions and this isn't one of them .

The above option would enable an external program to delete old logs after a certain time.

Even better would be an internal option to delete old log files after X amount of days or after Z amount of disk space is taken.

Cheers
Andy

Scheduler

andrew bear — Mon, 12 Jan 2026 15:44:48 -0000

Hi all

It would be nice to incorporate a scheduler to run any script or single sequence at any date/time or period. This would include the cyclic sequence facility.

Cheers
Andy

Cyclic Sequence: Forever + Pause units + Save + Simultaneous

andrew bear — Mon, 12 Jan 2026 15:38:42 -0000

Hi all,

One option not provided for the Cyclic Sequence that would be very useful is repetitions forever.

I do acknowledge this can be done with worker scripts but this is rather complex for something that seems simple and likely could be easily implemented in the cyclic sequence.

The easiest way to implement this would be to change the "Repetitions" field (and maybe appropriately renamed) so "0"=continuous, "1"= run once" , "2"= run twice, and so on..

The other option mentioned in the subject is pause units such as a dropdown for secs, mins, hrs etc which is convenient when polling for data over long periods of time.

in regards to the "Save" in the subject line often the same cyclic settings are used over time. So therefore it would be useful to incorporate the cyclic settings in the single sequence table.

In regards to "Simultaneous" in the subject line enabling simultaneous execution of sequences would be ideal. I did observe that this is what worker threads can be seemingly used for but this requires script programming and cyclic sequences don't. I acknowledge this isn't that simple to implement , as the issues with multiple threads and queues needed for this would have to be dealt with. However it seems most of this work has already been done for worker threads so maybe combining both would be the way to go if this is implemented.

Lastly the cyclic sequence option may be better incorporated in a scheduler. More about that in my next post.

Cheers
Andy

Add serialPort.read()

devnull — Fri, 19 Jan 2024 17:21:01 -0000

Agreed, i do it the same way, but using push

for (var i=0; i < data.lenght; i++) {
  rxbuffer.push(data[i]);
}

My hope hereby is that it more similar to a pipe. As the call to the sub is async, it might be that other subs reading from the buffer at the same time.

Add serialPort.read()

Stefan Zieker — Thu, 18 Jan 2024 07:12:53 -0000

Yes, the user has to take care of it. What I usually do is this:

function dataReceivedSlot(data)
{
    //Save the received data.
    g_storedReceivedData = g_storedReceivedData.concat(data);

    //Do something with data

  }

Recent posts to Feature requests and improvements

Logging - Dating of logs

Logging - Dating of logs

Logging - Dating of logs

Logging - Dating of logs

Logging - Dating of logs

Logging - Dating of logs

Scheduler

Cyclic Sequence: Forever + Pause units + Save + Simultaneous

Add serialPort.read()

Add serialPort.read()