Next: Monitors, Previous: Tutorial Introduction to fswatch, Up: Top [Contents][Index]
fswatchThis chapter is about how fswatch is invoked. There are
many options and two styles for writing them.
fswatchfswatch is invoked using the following syntax:
fswatch (options)* (paths)+
fswatch interprets file names as being relative to the
working directory and canonicalizes them using realpath.
If a directory is used as an argument, the directory object is watched and, optionally and depending on the monitor being used, the directory is scanned recursively and all its children are watched as well.
Depending on the monitor being used, recursively scanning huge directory hierarchies or big set of files may be resource consuming, CPU intensive or even impossible. The characteristics of the available monitors in a system should be assessed in order to choose the best monitor according to the specific needs.
Besides successful exits4, indicated with the exit
code 0, fswatch may exit with an error. fswatch
will try to print a diagnostic description on stderr when an
unexpected error occurs.
The documented5 exit codes of fswatch are the
following:
FSW_EXIT_OK: No error occurred.
FSW_EXIT_UNK_OPT: An unknown option was input.
FSW_EXIT_USAGE: Help message was requested.
FSW_EXIT_LATENCY: Invalid latency.
FSW_EXIT_STREAM: A stream related problem occurred.
FSW_EXIT_ERROR: An unkown error occurred.
FSW_EXIT_ENFILE: A file could not be opened.
FSW_EXIT_OPT: Unused.
FSW_EXIT_MONITOR_NAME: The specified monitor does not exist.
FSW_EXIT_FORMAT: The specified monitor is invalid.
fswatch implements two option styles which are common in
Unix and Unix-like operating systems and GNU software: short
and long options. The biggest difference between short and
long options are argument placing (for options taking one).
Whether long options are available in a system depend on the
availability of the getopt_long function at build time. For
this reason, users should familiarise themselves with short options
and use them when possible and do not rely on long options to be
available on any fswatch installation.
In systems where the getopt_long function is available, each
short option has a corresponding long option with a mnemonic
name starting with two dashes (e.g.: --version). Long
options are meant to be easy to remember and to provide hints about
what a command is going to perform. The following command:
$ fswatch --event-flags --numeric --recursive ~
is clearer than:
$ fswatch -xnr ~
If a long option takes an argument, it can be specified in two ways, depending on whether the argument is optional or mandatory:
$ fswatch --latency=5 ~
$ fswatch --latency 5 ~
Most options have a short form consisting of a dash followed by a single character, such as -l (which is equivalent to --latency). When available, a short form is interchangeable with the long one.
If a short option takes an argument, it can be specified in two ways:
$ fswatch -l 5 ~
$ fswatch -l5 ~
Short options can be stuck together provided all the options but the last one take no argument, in which case it can be specified as described above. The command:
$ fswatch -xnrl 5 ~
is equivalent to:
$ fswatch -x -n -r -l 5 ~
where ‘5’ is the argument of -l.
fswatch OptionsIn the following table you can find the list, in alphabetical order,
of fswatch’s options.
Monitor file access. This functionality is supported by selected monitors only.
Sets the allow overflow flag of the monitor. When this flag is set,
monitor buffer overflows are reported as change events of type
fsw_event_flag::Overflow.
Print a marker at the end of every batch.
Request the monitor to watch directories only during a recursive scan. This feature helps reducing the number of open file descriptors if a generic change event for a directory is acceptable instead of events on directory children.
Filter events by type using the specified event name or numeric mask (see Filtering by Event Type). Specified event names are included in the output. Multiple event types can be specified using this option multiple times.
Print the event flags.
Use the specified string as event flag separator.
Exclude paths matching regex.
Use extended regular expressions.
Load filters from the specified file.
Fire idle events.
Symbolic links are followed instead of being watched as file system objects.
Use the specified record format.
Print the event time using the specified format.
Show the help message.
Include paths matching regex.
Use case insensitive regular expressions.
Set the latency using the specified value.
List the available monitors.
Use the specified monitor.
Pass the specified property to the monitor (see Monitor Tunables).
Print a numeric event mask.
Print a single message with the number of change events in the current batch.
Exit fswatch after the first set of events is received.
Use the ASCII ‘NUL’ (‘\0’) as record separator.
Recurse subdirectories.
Print the event timestamp.
Print the event time as UTC time.
Print verbose output.
Print the version of fswatch and exit.
As seen in Observing File System Changes, file names may contain
characters such as ‘\n’ which are commonly used as line
separators. Many commonly used Unix commands and shell builtins use
characters in the $IFS environment
variable6 as
separators to split words. By default, $IFS contains the
characters ‘ ’ (SPC), ‘\t’, ‘\n’ and ‘\0’
(‘NUL’).
Therefore, if a file contains such a separator character (and all but ‘NUL’ are legal), then a parsing ambiguity may arise when using certain record formats such as:
path( flag)+
In this case, for example, if n > 1 flags are present in the record, and hence more than one ‘ ’ (SPC) is present, then it is not known whether any subset containing a number x of consecutive flags (x < n) is part of the path or not.
The same reasoning applies when splitting lines instead of words: since ‘\n’ may be a legal file name character, then it is now known whether ‘\n’ indicates a record’s end or simply is part of a file name.
For this reason, in order to avoid parsing ambiguity, this options
instructs fswatch to use ASCII ‘NUL’ as
record separator.
Warning: The use of the --print0 solves the line splitting ambiguity but not the word splitting ambiguity when using textual event flags. A solution to this problem is provided by custom record formats (see Custom Record Formats).
Another way to get an unambiguous record format is using numeric event flags (see Numeric Event Flags).
To solve the problem of line splitting ambiguities and to provide
users the possibilities of tailoring the record format to their needs,
fswatch allows users to specify the event record
format using the --format option.
This options requires a printf-like7
format string ordinary text containing zero or more
directives. Characters not belonging to a format directive are
copied unchanged to the output, while directives are interpreted and
replaced with the result of their evaluation.
Directives start with ‘%’ which is always treated as a special character: either it marks the beginning of a directive or it is interpreted as an escape character8.
The available directives are:
%%Inserts the ‘%’ character.
%0Inserts an ASCII ‘NUL’ (‘0’) character.
%nInserts a newline character.
%fInserts the list of event flags, separated by default by the space character (‘ ’) or by the separator specified with the --event-flag-separator option (see Event Flag Separator).
%pInserts the path.
%tInserts the timestamp, formatted with strftime using the
format optionally specified with the --format-time option.
Each record is terminated by either a newline character (‘\n’) or an ASCII ‘NUL’ character when -0 is specified. The record termination character has the following characteristics:
When the list of event flags is printed, textual items are separated by default by spaces (‘ ’). The user can specify an alternate event flag separator using the --event-flag-separator and passing the desired separator string as argument.
For instance, if the user wants event flags to be separated by a comma, the following command can be used:
$ fswatch --event-flag-separator=, -x (options)* (paths)+
The format used by fswatch when a custom format is not
specified is determined as follows9:
Since fswatch typically outputs an endless event
stream, processing parties parsing its output may be interested in
‘batch event processing’: that is, processing batches of events
instead of endlessly processing events one by one.
To support this use case, fswatch provides the
--batch-marker option; when specified, fswatch will
output a customizable ‘batch marker record’ processing parties can use
as batch delimiters. Batch demarcation is made naturally using
the monitor’s processing loop and its latency setting: every time the
monitor loops (typically when latency is elapsed), then a batch marker
is printed as final record, as shown in the next example:
$ fswatch --batch-marker -r ~ /home/fswatch/.zsh_history.LOCK NoOp /home/fswatch/.zsh_history.new /home/fswatch/.zsh_history /home/fswatch/.zsh_history.LOCK NoOp
By default, the batch marker takes the form of a single-line record:
NoOp(\n | \0)
terminated with either ‘\n’ or ‘NUL’ (‘\0’) depending
on other fswatch settings. However, the user can customize
it by providing the desired marker string as optional argument to
--batch-marker:
% ./fswatch --batch-marker="*** BATCH END ***" -r ~ /home/fswatch/.zsh_history.LOCK *** BATCH END ***
An idle event is a special event type that can optionally be
emitted by fswatch if no change events were collected in a
period of time whose average length is equal to the monitor latency
(see Latency). Idle events come in handy when an observer wants
to perform an operation every time a change is detected or
after a specified amount of time even if no changes were detected.
Idle events were introduced in version 1.9.0 and are available only
when fswatch is built on a platform that supports C++11
threads (std::thread) and can be enabled using the
--fire-idle-events option.
An idle event has got the following characteristics:
NoOp.
Filters are regular expression which are evaluated against the monitored object path to determine whether a path must be accepted or rejected. Sometimes, the exclusion of a path may result in the exclusion of an object from the list of monitored objects, while other times a path must be evaluated only when an event is detected and in this case the corresponding object cannot be removed from the monitored object list in advance10.
Event though event filtering is commonly performed when
processing fswatch output, the possibility of filtering
paths ‘at the source’ provides not only a greater amount of
flexbility, but also:
fswatch will only monitor
matching objects11.
fswatch.
Filters are implemented using the C++11 <regex> library. This
feature is now required to successfully build libfswatch and
fswatch.
Path filters can be specified in two ways:
The filter file is made up of records separated by a new line charactere, or formally a ASCII ‘LF’ (line feed) character. The structure of the record is the following:
type pattern
where ‘type’ indicated the filter type and ‘pattern’ is the filter regular expression. ‘type’ may contain the following characters:
The following filter file instructs fswatch to ignore all
files except those ending in ‘.cpp’, ignoring case.
- .* +i \.cpp$
Two types of filters are available:
As their name indicates, they are used to include and exclude paths
from the monitored object list and from resulting events.
fswatch processes filters this way:
Said another way:
Filters are regular expression executed using the
regcomp function which is able to interpret case-sensitive
and case-insensitive basic and extended regular
expressions as described in
Base Definitions volume of IEEE Std 1003.1-2001,
Chapter 9, Regular Expressions.
The (--insensitive, -I) option instructs
fswatch to use case insensitive regular expressions. The
following example adds an exclusion filter so that fswatch
ignores any file system object whose name ends with .log, no
matter the case.
$ fswatch -Ie ".*\.log$" ~
The (--extended, -E) option instructs
fswatch to use extended regular expressions, such as:
$ fswatch -Ee "xl[st]+" ~
Treating the characteristics and the difference between different kinds of regular expressions is out of scope in this manual.
Events can be filtered by event type passing fswatch a list of
event type names or masks to accept using the --event
option:
$ fswatch -x --event Created --event Removed ~ $ fswatch -x --event 10 ~
In this example, the mask of the Created event is 2
and the mask of the Removed event is 8, so the mask
of both events is 10.
If no event type filters are specified, fswatch will accept
events of any type; on the other hand, as soon as a filter is specified,
only events with a matching type will be accepted.
The latency l, expressed in seconds, is the amount of
time that passes between the moment fswatch outputs a set of
detected changes and the next. What happens during the time
in-between is a monitor-specific implementation detail.
Some APIs, such as OS X’s FSEvents, implement the concept of
latency themselves and fswatch appears idle in between.
Only when the specified amount of time passes, change events are
received, processed and written to standard output. Others, such as
Linux’s inotify, do not12; in this case, the inotify monitor waits
for events a maximum of l seconds; after that, the monitor
logic loops again, performs house-keeping activities13
and starts waiting again.
The important thing to keep in mind is that latency and a monitor’s behaviour are implementation-dependent: check the documentation of the monitor you are using to get further information about how latency is handled.
Symbolic links are commonly used file system objects and, as it is
customary for file system utilities, fswatch can either
follow them and monitor the linked object14 or monitor the link
itself.
Event flags identify the kind of change a file system object has undergone. Many of them directly map to common file system operations (such as creation, deletion, update, etc.), others are less common (such as attribute modification), and others are monitor and platform specific.
Currently, fswatch maps monitor-specific event flags to
‘global’ event flags acting as a sort of ‘greatest common denominator’
of all the available monitor flags. The list of all the available
global event flags, defined in c/cevent.h, is the following:
NoOpIdle event, optionally issued when no changes were detected.
PlatformSpecificThis event maps a platform-specific event that has no corresponding flag.
CreatedThe object has been created.
UpdatedThe object has been updated. The kind of update is monitor-dependent.
RemovedThe object has been removed.
RenamedThe object has been renamed.
OwnerModifiedThe object’s owner has changed.
AttributeModifiedAn object’s attribute has changed.
MovedFromThe object has moved from this location to a new location of the same file system.
MovedToThe object has moved from another location in the same file system into this location.
IsFileThe object is a regular file.
IsDirThe object is a directory.
IsSymLinkThe object is a symbolic link.
LinkThe object link count has changed.
OverflowThe monitor has overflowed.
As you can see, the list of event flags contains element whose meaning
is overlapping, at least partially. Link, for instance, may
be equivalent to Create or Removed, depending on
the whether the new link count is 1 or 0. MovedFrom and
MovedTo may be equivalent to Create and
Removed if the monitor is unable to discern a move operation
has taken place (which is not always possible, as in the case of the
poll monitor).
fswatch is unable to univocally map the specific flags of
all the monitors consistently. Forcefully, the mapping depends on the
capabilities of the monitor which, in turn, depend on the capabilities
of the API being used.
For this reason, when processing change events, either the behaviour of the underlying monitor is known and taken into account, or all the flags which could possibly be attached at the operation being looked for must be taken into account.
Warning: As already explained (see Whitespace and Record Format), the record format when using event flags in textual form is ambiguous. For this reason, using numeric event flags (see Numeric Event Flags) or a custom record format (see Custom Record Formats) is recommended when
fswatchoutput must be processed.
When using the (--numeric, -n) fswatch
will output event flags in numeric format. A change event
record may have multiple event flags and the numeric value is
calculated as the bitwise or of the numeric values of all
the flags. Since the value of an event flag is guaranteed to be
unique and to be a number n in the form n = 2^k for a
certain integer k, then the numeric value of a set of event
flags is univocally determined.
To check whether a given event flag is present when processing
fswatch output, iti s sufficient to check whether its bit is
set to 1 in the event value. Let’s suppose we want to check whether
the event flag whose value is e is present in a record whose
flag numerical value is n. If the result r of
where \wedge is the bitwise and operator, is r
> 0, then the flag e is present in n.
The numeric value of all the event flags is the following:
NoOp: 0
PlatformSpecific: 1
Created: 2
Updated: 4
Removed: 8
Renamed: 16
OwnerModified: 32
AttributeModified: 64
MovedFrom: 128
MovedTo: 256
IsFile: 512
IsDir: 1024
IsSymLink: 2048
Link: 4096
Overflow: 8192
fswatch is a front-end to multiple monitors, each
taking advantage of different monitoring APIs that may be
available in a system. When building fswatch,
configure scans the system to check which APIs are
available and builds support for all of them.
A ‘special’ monitor, the poll monitor, manually scans the file
system looking for differences. This is a fallback monitor for
situations where other, more efficient APIs are not
available. The poll monitor is available on any system providing the
stat function.
Although fswatch chooses the ‘best’ monitor between the
available ones, a user may wish to use another. A specific monitor
can be chosen using the (--monitor, -m) option. The
list of available monitors can be obtained using the
(--list-monitors, -M) option or at the end of the
help message:
$ fswatch --list-monitors fsevents_monitor kqueue_monitor poll_monitor
$ fswatch --help [...] Available monitors in this platform: fsevents_monitor kqueue_monitor poll_monitor [...]
A monitor can then be chosen by passing the mandatory ‘name’ argument to the -m option:
$ fswatch -m kqueue_monitor ~
In this case, the ‘kqueue_monitor’ is manually chosen.
fswatch’s behaviour when scanning a directory may vary on a
monitor by monitor basis. The semantics of the (--recursive,
-r) option is: recursively scan subdirectories. However,
implementations may silently add ‘if the monitor does not do so
already’. Since each monitor uses a different API, its
behaviour depends on that of the backing API, and it is
monitor-specific.
fswatch will walk the file system hierarchy rooted at the
directory and will open a file descriptor for each children to
establish a watch on it.
ReadDirectoryChangesW API
returns change events for first-level children of a directory. When
the -r option is not specified, change events for a watched
directory’s children are received. When the -r is specified,
the monitor will walk the file system hierarchy rooted at the watched
directory and will establish a watch on every directory object found.
In general, users should always use the -r option according to its semantics, no matter what the monitor does. The only case when -r is ‘not’ honoured is when a monitor adds information by recursively monitoring children even when -r is not specified. Please notice that when this happens, there may be no performance overhead since the backing API is specifically designed to behave like this.
The authors think this is not a problem. If you think this behaviour can be improved, please fill a bug report (see Reporting Bugs and Suggestions).
Some monitors such as the kqueue monitor require a file
descriptor to be open for each watched file system object. This
imposes a limitation on the maximum number of files that can be
watched by fswatch. Before version 1.7, a user could only
overcome this problem by increasing the maximum number of open file
handles on its system.
fswatch 1.7.0 introduced a new option,
-d/--directories; when this option is used with a
monitor that supports it, only directory objects will be watched
during recursive scans. When a change occurs on a file, instead of
reporting which file has changed and how, fswatch
will report a change event on the parent directory: this way, the
number of required open file handles decreases at the expense of
change event information granularity.
Some monitors may accept monitor-specific parameters to tune their
behaviour. To this purpose, fswatch offers a mechanism to
pass key-value pair which are literally passed to the underlying
monitor. A key-value pair (k, v) can be passed to a monitor
using the --monitor-property option:
$ fswatch --monitor-property k=v ~
Multiple key-value pairs can be passed by using the
--monitor-property option multiple times.
Depending on the monitor and options
being used, fswatch may not exit unless stopped with
a signal such as TERM or QUIT.
Exit codes are documented in c/error.h
of libfswatch.
IFS (Internal Field Separators).
Although the
available directive are much less than what printf offers.
Which is the same as considering escaped characters the result of a directive.
In the following example, the record termination character is not shown.
This behaviour is monitor-specific.
Whether an object whose path is matched by an exclusion filter is monitored or not is a monitor-specific implementation detail.
inotify publishes changes on a file
identified by a descriptor which is read by
fswatch.
Such as re-scanning objects which did not exist in the previous iteration.
When following links, the resolution is recursive: that is, if a link points to another symbolic link, this link is followed as well, and so on, until an object of a different kind is found.
But
manually filtering out events based on paths, but fswatch
does not do so by design.
Next: Monitors, Previous: Tutorial Introduction to fswatch, Up: Top [Contents][Index]