pngquant 2.12 (#40743)
Release date: 2018-08-08
FIT 14.6.11 is mainly a maintenance release comprising bug fixes and third party updates.
However, developers will find some smaller improvements that make it easier to create custom requests for REST or SOAP APIs.
must-revalidatedirective and ignores all
Cache Control Extensions(#40909)
304response that forbids caching (e.g.
Cache-Control: private) upon revalidation, the cache entry in question will be deleted (#40454)
typeattribute, the JSON XML notation element
<json-document>now defaults to object type (
jsonautomatically converts JSON XML notation before sending (#40972)
POSTan inline XML document without escaping (#40972)
xml2jsonconverts the JSON XML notation into JSON text while reading from a resource (#40972)
responseXML now includes a
methodattribute reflecting the HTTP method used for the request (#40856)
fit://request/cookiescould result in doubled cookies when the
pathfield was missing (#40301)
client/versionfor SamsungBrowser now reflects the Chrome version number (#41202)
Release date: 2018-06-06
14.6.10 introduces new XPath functions for developers working with JSON data.
lookup() function implements key based lookups from either JSON files or our open sourced SHMT database format. Especially the latter is very fast at runtime. The function’s purpose is to perform pain free “value-value” replacements from XSL transformations such as mapping IDs to strings or implementing translation tables.
The returned value of
lookup() may be an arbitrary string, such as JSON data. This plays nicely together with the new
xml-to-json() functions that are loosely based upon the new XPath 3.1 functions for handling JSON data. They translate JSON strings into an XML representation that is returned into the XPath or XSLT context as a ready-to-use DOM tree. This is especially handy when working with API services.
As usual, there are also a couple of third-party updates and bug fixes.
performance.timingis now added to the FDX
fit://request/cookiesto allow for sending updated cookies to the client. (#40301)
fit://request/cookiesmay be set
0(i.e. the UNIX epoch) to signal the browser to delete the cookie (#40301)
Release date: 2018-04-04
14.6.9 is a maintenance release comprising bug fixes and third-party updates.
Image Compression produces smaller WebP results from PNG input files with few colors. Such files often carry graphical images that are more susceptible to compression artifacts. However, even natural photos may be quantized to a small color palette. Instead of relying on WebP lossless compression to avoid artifacts, a lossy compression with higher quality is carried out. This saves a lot of bytes for photo-like images while preserving quality for logos and icons.
Release date: 2018-02-07
14.6.8 is a maintenance release with improvements, bug fixes and third party updates.
For the folks operating FIT servers, we are happy to finally ship a solution for the issue with excessive memory use due to brotli compression. It turned out that neither
libbrotli were leaking memory. However, the way
free()s leaves behind a fragmented heap that
libc is not willing to return the OS. With the right parameters, we could tune the allocator to avoid this memory bloat, reducing memory usage by 500 MB per
httpd process on Red Hat based systems (and up to 100 MB on Ubuntu or SLES).
From time to time, one needs to clear the HTTP cache, e.g. when resources in the origin change. Instead of deleting all cached files,
clearcache --http now sets a reference file that instantaneously invalidates all files. The origin then may send lightweight
304 responses to revalidate the existing caches.
After another round of data crunching we have tuned the parameters for our image quality prediction algorithm in order to obtain smaller files with
Automatic Image Quality Assessmentto obtain smaller images (#37049)
s-maxage=0directive to speed up revalidation and to allow optimizations, e.g. the Image Compression (#35466)
no-cache="Set-Cookie"directive are now cacheable by the HTTP Cache (#39077)
0in the CSS Minifier (#38953)
CSS concatresulting in parts of the CSS code being ignored by browsers (#39300)
low-speed-timemay be set via
mallocto workaround a memory bloat issue with mod_brotli caused by heap fragmentation (#39024)
Release date: 2017-12-14
126.96.36.199 is a hotfix release for FIT 14.6.7.
The updates of Apache
mod_brotli and Brotli 1.0.1 both had memory leaks resulting in excessive memory usage of
httpd processes. These third party libraries have been reverted to the versions shipped with FIT 14.6.6.
mod_brotliand Brotli (#39024)
Release date: 2017-12-06
FIT 14.6.7 rounds up this year’s line of releases with a long list of improvements. We have touched all Web Acceleration components in order to improve run-time performance, compression efficiency and standard compliance. In short: we do a better job in less time!
We have a passion for Image Compression and constantly research new ways of improving our compression results. A new detection step in our Image Compression identifies redundant transparency in image files. If the alpha channel of a
PNG file marks every pixel as opaque, the channel may be removed altogether. This alone may lead to smaller file sizes. However, the main benefit of removing transparency is the possibility to achieve much higher compression ratios by using the
JPEG format. Thanks to greyscale support and better quality prediction, AIQA can asses more
JPEG candidates, leading to smaller file size.
At the same time the CPU needed for image processing was reduced. One seemingly simple task is counting colors. Larger images may contain millions of distinct colors. However, in many cases the exact number is not necessary. Taking a short cut when counting colors can cut processing times from seconds to milliseconds!
We have also improved our markup processors. The HTML Minifier runs much faster as it is now completely written in C. HTML5 parsing with Gumbo is faster, too. Also, its parsing results are better, because we have implemented a server-side version of
noscript handling. HTML5 specifies parsing rules for scripted and non-scripted contexts. However, when pre-processing HTML documents the client’s execution context is unknown. Therefore we handle
noscript in a way that, while actually parsing its content, has only local effects on the DOM structure. This is especially important if
noscript blocks in the document’s
head contain palpable content. Reordering
head elements now also handles Internet Explorer’s Conditional Comments. The
Minifier handles redundant HTML comments better and provides “bug compatibility” for IE problems.
Besides quite a few bug fixes we have updated many bundled third party software to keep you safe over Christmas :)
noscriptoption for the HTML Minifier to
falsedisables the minifying for
Cache-Control: no-cacheresponses to speed up revalidation in subsequent requests (status 304) (#37763)
post-checkare now considered to be cacheable (#38529)
s-maxagedirectives are now considered to be cacheable (The shortest time to live will be used) (#38289)
Content-Typeheaders in 304 backend responses that differ from the original header (#38417)
privateis added to the
noscriptelements inside the
Image Compressioncache for local images (#38412)
Dynamic Parameters and Optionsin the
regexaction now relate to the actual cause of the error (#38307)
fitadmin config checkchecks whether exclusive file locking works (#37640)
Release date: 2017-10-04
14.6.6 is an optimization release comprising quite a few improvements and bug fixes. You will find improvements to many of our acceleration features.
The HTML Minifier runs faster to bring down TTFB. The HTTP Cache supports
proxy-revalidate to satisfy more responses from cache.
We have removed on-the-fly Image Compression. That means an image that is not cacheable will not be processed at all. Until now, we would perform a compression experiment without storing the result. However, we have decided to trade TTFB for download size.
Our HTML5 parser now fully adheres to the W3C Recommendation to detect the encoding. Most of the time, the character set is determined from a byte-order-mark or the
Content-type header field. If neither of those are present, our new scanner reads the encoding from the
<meta> tags in the document itself.
As usual, the update ships fresh 3rd-party libraries to carry on upstream security and stability fixes.
Content-Typeheader field, the HTML5 parser now uses the algorithm specified by the W3C Recommendation to detect the encoding (#37364)
proxy-revalidatedirective, are now cacheable by the HTTP Cache (#37589)
clientTimingnow contains the property
navigationTypein case the browser provides the
windows.performance.navigationAPI. Possible string values are
Cache-Control: private) responses. Such responses are passed unchanged to the client and will never be processed. (#37061)
enctype="multipart/form-data"set now contains the correct submit button information when sent via PPL (#37322)
srcattribute in case a mandatory URL map is used (#37485)
fitadmin --version --shortprints the clean version string like
Release date: 2017-08-02
Doing nothing at all is better than doing something fast. That’s what caching is all about. This release has a lot of features to avoid unnecessary work. FIT has had a HTTP Cache for a long time. But when the time-to-live of a cached resource has passed, the content had to be requested again. While it is mandatory to contact the origin server again, it is not necessary to actually download an unchanged resource. HTTP provides conditional requests and the status code 304 for an intermediary cache to check whether the local copy a file is still valid. In many cases this check is much faster than a fresh download. Especially for short-lived resources this can greatly reduce bandwidth use and network traffic for origin servers.
There is also CPU time to save. FIT carries out CPU intensive work on image files. System operators now have better control how the disk cache is managed. The new option
CACHE_DISK_MIN_FREE_PERCENT avoids premature cache cleanups. As long as the regular automatic check finds enough free disk space, no files are deleted. Only if the configured threshold is exceeded, files are deleted. If you have enough disk space, this can reduce CPU usage significantly by eliminating repeated calculations.
Sometimes it is necessary to manually purge a cache. For example, if a resource changes but its URL stays the same. The
clearcache command now offers to delete only parts of the cache. For example
--http clears the request cache but leaves the precious Image Compression cache untouched.
With improved handling for character encoding, we take the HTML5 parser out of beta. The parser, based on Google’s Gumbo, has proven its stability and conformance.
Besides that, 14.6.5 brings fresh 3rd-party libraries, shorter TTFB for image requests and bug fixes.
Content-Typeheader field (
text/plain;charset=UTF-8, as with FDX beacon requests) in order to satisfy security checkers (#37063)
srcsetattribute values (or custom attribute values declared to have the same pattern as
srcsetattributes) are properly rewritten if image scaling is disabled and responsive image filtering is enabled (#37279)
FormDatanow include the data of the button used to send the form (#36402)
&in Dynamic Attribute Values (#37186)
CACHE_DISK_MIN_FREE_PERCENToption to skip cache maintenance as long as sufficient disk space is available (#36686)
fitadmin maintenance clearcache(#36053)
Release date: 2017-06-07
With this release we are proud to take our Automatic Image Quality Assessment out of beta. We have made three significant improvements to this Web Accelerator module. In order to find the highest acceptable compression ratio, AIQA ensures that the visual quality is not degraded. Therefore it is essential that the assessment score closely correlates with human perception. The SSIM algorithm provides good results in many cases. However, especially commercial product images often have a very uneven distribution of image details. Consider a cut-out, in-focus product framed by a passe-partout. Conventional algorithms tend to dilute the errors introduced in complex image areas with the “perfect reproduction” of large background areas.
We have created a Saliency Weighted variant of the SSIM algorithm that is almost immune to this kind of overrating. For example, enlarging the monochrome backdrop around a product should not change the calculated image distance introduced by distortion. Images in natural settings benefit, too, by correcting the weight of errors in background areas. This produces very accurate distance values that help us minimize visible compression artifacts.
To ease the use of this technique, we have compiled a set of Compression Presets that makes configuration as simple as deciding between
quality – or using the
balanced default values.
Image Assessment is a very CPU intensive task, because finding the optimal compression settings usually involves multiple compress-and-assess cycles. We have analyzed the compression characteristics of thousands of images in order to train a prediction model. It estimates near optimal settings after a single cycle with a very low reject rate.
For developers we have created more possibilities to control HTTP requests: Super fine-grained timeouts and origin response header injection. The latter can be used to correct erroneous or provide missing
There are a lot of smaller improvements, bug fixes and third party updates, too. The changelog provides a thorough list.
%literals no longer cause the request to be aborted (#36089)
Cache-Control: privatein addition to
Cache-Control: no-cache, no-storein order to avoid problems with some CDNs (#36703)
response-headerparameter to remove, set or replace the incoming HTTP header fields (#33431)
low-speed-timeto abort downloads that are too slow. (#32826)
ai-scalingparameters are now ignored unless they specify positive decimal numbers with
pxas their units. (#36047)
Content-Typedetection for local files (#34901)
<cache enable="false" />now works as documented and deactivates both the HTTP and the Interval cache (#36187)
Bad FIDJ URI: <URI>is removed from the debug output (#36455)
Release date: 2017-04-05
April’s 14.6.3 release comes with a host of new tools for developers. It is now much easier to deal with JSON backends with the
<json-parser> option. If set, the
parse action will convert a JSON response into a DOM representation for easy manipulation. In case the backend sends an HTML response, the parser is chosen accordingly. When accessed from the
document() function, JSON content is now always parsed into a DOM.
The new XPath functions
fit-serialize() are handy for debugging and for instrumentation, e.g. feeding run-time messages into a logging and monitoring chain.
Improvements of the acceleration features are centered around image handling. We have added support for ICC profiles for JPG and PNG images, the performance of image quality assessment is much better, there are improvements for image delaying and rewriting of image URLs in CSS code as well as fixes for image scaling.
To avoid manual configuration of URL encoding options, we have changed how the
default-request handles query strings. Whenever possible, the incoming query string is forwarded unchanged to the backend.
content-handling/json-parserautomatically converts JSON to XML (#35498)
<connection reuse="true | false" />request option can be used to disable HTTP persistent connection (HTTP keep-alive) for individual sources (#35577)
force-query-rebuildoption for the
fit-documentand the XSLT function
documentautomatically convert JSON data to XML, if appropriate. (#35498)
ai-…comments in HTML
height) are ignored (#35757)
User-Agentheader value containing
Windows NTresults in
default-requestnow uses the unchanged query string whenever possible in order to avoid problems due to changes in URL encoding (#34899)
ai-priority= 0|1|2|3is not overwritten with
visibilityto ensure images in hidden container can be delayed but are loaded before the container is shown. (#35707)
src attribute. (#34078)
overflow: hiddenare now loaded correctly (#35798)
liblcms2-2on SUSE and Ubuntu). (#34429)
fit.inican be used to completely disable HTTP persistent connections (HTTP keep-alive) for all origin servers (#35577)
fit.inican be used to set a default connection timeout for outgoing HTTP requests (#35868)
Release date: 2017-03-08
FIT 14.6.2 focuses on stability and security.
We have replaced our existing CSS minifier by an entirely new implementation. The CSS parser provides better standard compliance and is a lot faster, especially for CSS files with large embedded images. On top of that, the minifier produces smaller results. As part of our open source effort we have made our PHP7 CSS3 extension available at https://github.com/sevenval/php-ext-css.
We have improved system stability with a couple of new knobs for admins. Image compression refuses to optimize images that are larger than the configured maximum area in order to avoid excessive memory and CPU usage. Furthermore, we have added a new
fit.ini setting that limits the number of concurrent image optimization processes. This is an effective means to protect your cluster from image compression overload, e.g. caused by an in-rush after a cache flush.
Last but not least, 14.6.2 is the first release to be shipped with PHP 7.1.
Set-Cookieheaders when serving a cache hit (#35669)
force-client-httpsredirects HTTP client requests to HTTPS with status
301 Moved Permanently(previously
FIT_ALLOW_OPEN_ACLcan be used to override the
allow-all="true"option of sites’
FIT_MAX_IMAGE_PROCESSESto limit the number of concurrently running image optimization processes. (#35258)
Release date: 2017-02-08
FIT 14.6.1 is the first release for 2017. And, while being packed with lots of bug fixes and improvements for existing features, it even brings a few new features for both, development and administration.
First off, we added the function
ends-with to our stack of XPath functions.
starts-with was already accessible everywhere in FIT where an XPath expression is supported, adding
ends-with rounds off the set of string checks available.
For operation and administration we also have 2 new features. We now officially support
systemd, so you can use the native init management tools in state of the art Linux systems. And to prevent huge content from being processed and overly stressing your CPU resources we added the
FIT_MAX_CONTENT_SIZE. With that you can specify a maximum size (in
MiB) for content processing. When a resource exceeds that size, it will not be processed, but passed through unoptimized.
On the improvement front, we worked a lot on our image optimization.
The image compression engine, that was introduced in FIT 14.6.0, will now perform the optimization experiments after the actual requests have been handled. So when an image is requested, FIT will respond with either the original or the best experiment result so far. After the response is sent to the client, FIT will run its experiments, so the next request can be responded with an optimized version of the image, while neither user had to wait for an experiment to finish. This only works for images that are cacheable by FIT. Images that may not be cached are still processed live.
composeImageCompressionUrl) or image scaling (
composeImageScalingUrl) URLs. And the already established function
composeImageUrl will now create URLs according to your config settings.
Image delaying received some updates, too. While we now opt-out of delaying for
img tags within, for example,
template tags, FIT is now able to delay images that use the
On a side note, we tweaked the performance of many features, from analytics over image processing up to resource loading.
srcsetattribute can now also be delayed. (#34447)
templatetags are excluded from any image delaying. (#34551)
composeImageUrlnow respects the config settings for image compression and image scaling. We also added
composeImageCompressionUrlto create a scaling URL or a compression URL regardless of config settings. (#34816)
fit://localURLs can be rewritten without an URL map (#34517)
styleelements of enabled adaptive components are now inserted into the document’s
headright in front of the first element with the same node name, such that the order of existing
headelements remains unchanged. (#25202)
Blinkas render-engine. (#34809)
max-stale– after expiry, one request will revalidate the resource while all others wait (#34659)
systemdsupport for PHP and Apache on Red Hat/CentOS systems (#33053)
fit.inilimits the input object size to prevent excessive CPU usage by preventing optimization of very large resources (#34547)
Release date: 2016-12-07
This year’s last release is a big one. FIT 14.6 introduces our next-generation Image Compression engine.
To find the optimal representation for an image, the Image Compression engine performs a number of optimizations suitable for the input image. For example, lossy compression techniques (WebP/JPEG) may be applied to a photograph for substantial size improvements. Transparent PNGs may benefit from ZorroSVG mapping, or graphics from WebP lossless coding or quantization. The experiments are traffic-aware, so that frequently requested images are taken care of with higher priority.
<image-compression> can be used if Image Scaling is disabled. As a bonus, the handling of Image Compression URLs in HTML (and CSS) documents is much faster, which reduces the footprint of HTML processing in the critical path.
As a BETA feature, we publish an automatic image quality assessment tool. Lossy compression usually achieves the highest byte savings. However, it is impossible to find a compression level (ironically often called “quality”) that balances compression and quality for all images. Different images show different patterns of degradation if the compression level is increased. While some images still look good with JPEG “quality” 30, other images may start showing visible artifacts already at high levels like 90. The common solution is to raise the quality for all images. You can now define the compression level by means of quality: “Compress an image as much as possible, as long it looks like the original”. This can be a real byte-saver! It allows us to apply lossy compression to input images in lossless formats without the risk of quality degradation. As this is beta, we would love to hear your feedback about this!
Another interesting beta feature is the HTML5 compliant parser. It parses HTML documents exactly like your browser, and you can still use all FIT development features like XSLT and XPath.
As this is a major release, the various bug fixes, 3rd party updates and smaller improvements are accompanied by a couple of removals of previously deprecated features. So please pay close attention to the changelog.
redirectsinternally, the HTTP Cache now ignores headers of intermediate responses to determine the cacheability of the latest response (#33620)
add-size-attributescalculation now ignores relative size specifications instead of treating them as absolute values to avoid incorrectly sized images (#33965)
fit://localallows to use absolute URLs for local sources (#34250)
client/fit/cdridnow has only 1 part (representing the browser) instead of 3 parts (representing hardware, operating system, and browser) (#34326, #34101)
client/hw/display/widthnow depend on the device’s orientation (#34326)
imagetags into HTML
visibility, resulting in all images being visibility-delayed. If
ai-priorityattributes should be respected,
prioritization="auto"must be set. (#33190)
Release date: 2016-11-03
In addition to fixing bugs FIT 14.5.3 introduces new and improved Web Accelerator features.
In many modern projects, iframes are used for social media widgets and user tracking. They usually have no impact on the perceived performance of a web page. But there are also sites that have iframes with heavy-weight media content such as youtube videos. Loading megabytes of videos slows down page loading tremendously, even in iframes.
To ensure that users will only load content they are about to see, we introduce delayed iframes. The delayed loading for iframes works exclusively with visibility based loading: The source of the iframe is going to be loaded when the iframes comes close to or into the browser’s viewport, for example by scrolling the page. Due to the unknown nature of the included content of iframes (small tracking and social media vs large videos), the feature cannot be activated globally. You have to enable it in
config.xml and mark the individual
iframe element with the
Probably even more exciting is an improvement we made for delayed images. If the dimensions of images are already known (i.e. the image is in the FIT cache) FIT will generate an SVG image with the same intrinsic size as the final image (instead of a transparent 1x1px GIF). Hence, the layout of the page will remain unchanged when the final image is loaded and replaces the placeholder. This saves a lot of browser reflows and avoids “jumping pages”. The SVG placeholder also clears the way for more promising improvements to the delayed image placeholders, so stay tuned for further news on the topic in upcoming releases.
delayed imagesare now delivered as inline SVGs to improve page rendering (#33303)
webfont-compressionnow also creates WOFF or WOFF2 fonts from TrueType fonts. (#33553)
Varyheader fields (#33301)
advanced-cache-control low-precision-viewport="true"feature is no longer activated implicitly by
script-managerfeature, causing scripts to be evaluated in the wrong order, was fixed. (#33851)
fitdetectioncookie is present, the viewport size DC properties now correspond better with the hardware size and the device pixel ratio. (#33319)
301to avoid losing POST request parameters. (#34099)
Release date: 2016-10-05
FIT 14.5.2 is another release bringing mostly bug fixes and improvements for existing features.
Delayed images prioritized to be loaded when they are going to become visible considerably reduce traffic. But if visitors want to print sites, they are basically missing every image that neither is nor has been in the viewport. The new parameter of the
loadHQImages function makes the client load all images, whether they are in the viewport or not. And the new event
delayedimagesloaded informs you when the client finished doing so. So this combination offers means to prepare the site for printing and lets you know when preparing has finished.
/ as relative URLs. So when you write something like
<script>var foo = '/bar';</script> into your document, it will not take long until you notice said spiders requesting the resource
Additionaly, fixing issues with the image scaling cache has reduced the CPU usage noticeably.
delayedimagesloadedis fired after all delayed images have been loaded. (#32720)
scriptelement itself, was fixed. (#25861)
remove-duplicates="true"will no longer break if the
linkelements contains uppercase letters. (#33410)
DOMParsernow correctly parses in Internet Explorers with delayed images activated or AC-Stage initialized. (#33505)
Image Scalingcache entries, such that they are not expired immediately. (#33594)
Image Scalingby correctly setting the encoding filter and compression level. (#33304)
query-encodingoption can be used to specify a list of characters that are not to be percent-encoded in the query string. (#30863)
query-encodingoption instead. (#30863)
Image Scalingby eliminating dimensions from the URL if the image will not be scaled. (#33232)
304are now logged as
mainrequests in the
log files. (#33598)
Release date: 2016-09-07
This release is mostly a “smoothing” release comprising a lot of bug fixes and smaller improvements, as well as 3rd party updates and security patches.
One of the most common errors that we have encountered in our log files was a complaint about source images being too large:
… image area exceeds limit of 8 Mpixels. Of course, 8 million are a lot of pixels. But with the proliferation of high resolution cameras (in smartphones) and high density displays (also in smartphones), prepare to encounter large images in Web sites more frequently. However, the worst part of the error message was:
Continue with original
source. That means that of all things the largest images on your Web site are “too large to optimize”. The client has to load all these big files as they are. Even if those might be the outliers of your image files, a single large image with megabytes of data will outweigh every little byte saved by thoroughly optimizing everything else.
Acknowledging this problem, the Web Accelerator now supports sources images with up to 100 mega pixels.
Image Scaling will drastically reduce the file size of large images by allowing optimizations such as format conversion or lossy compression. On our systems, we have seen savings of multiple mega bytes per image. To mitigate the operational costs of processing huge pixel arrays, images with more than 8 mega pixels are processed with an algorithm that is less precise but faster than our usual scaling method. Given so many pixels, there is no noticeable difference.
Don’t let the legendary Bigimage eat your bandwidth! Update today!
brotlicompression would typically enlarge them (#33181)
headelement (e.g. if first tag is
link) as complete documents rather than fragments (#32611)
resolutionmedia queries is only performed if the device pixel ratio was detected (#32419)
lm-marks that would normally be cacheable for long durations are now flagged as uncacheable if their
Last-Modifiedheader indicates content changes (#33036)
WOFFbefore converting it to
WOFF2(other file types will be passed unchanged) (#33298)
Content-Typeheader are passed to the client. This ensures that the boundary is not lost for
hqimageloadedevents, which may have been fired multiple times for each delayed image, are now fired only once. (#32674)
;or in combination with PPL or the AJAX API (#33346)
viewport/resizeableis now considered deprecated (#32419)
server/cdr-versionis removed, due to the fact that the Client Description Repository (CDR) is no longer released separately from FIT. (#32813)
/var/cache/fit14/tmp) is created on-the-fly, if missing (#32925)
fit_request.logmay have been incorrect for certain timeouts and other errors (#33034)
Release date: 2016-08-03
FIT 14.5 is a feature release for the Web Accelerator.
Brotli is a new data compression library that shrinks Web payload better than proven methods such as
deflate. The Web Accelerator now includes brotli as an Apache module. We have seen an improvement of over 10% of the average compression ratio compared to gzip. This means that compressible payloads (almost all formats except images) are transferred to clients in less than a third of its original weight.
Brotli also accounts for much of the benefits of the WOFF2 font format. Those font files load significantly faster than the predecessor format WOFF or the old TTF format. The Web Accelerator beta feature
webfont-compression converts WOFF fonts on-the-fly into the WOFF2 format, resulting in 20% or more reduction in payload. As fonts are usually loaded in the critical render path, this is a welcome diet. To keep things simple, the Web Accelerator handles the necessary changes to your CSS font definitions as well!
To further accelerate loading of CSS files, we have improved our
style-concat feature to work with any external CSS files. All adjacent
<link rel=stylesheet> definitions within the same media query selector are combined into a single request. This not only frees up request slots in HTTP/1 setups, it also allows for more efficient data compression than single requests would have. Furthermore,
style-minifying now removes even more redundant code and has improved runtime performance.
In 1999 Microsoft gave birth to the infamous
favicon.ico. Of course everyone loves icons, but what made the
favicon so annoying was that it was a request the browser conducted itself without the developer referencing it in the page. That makes it so easy to forget. Later, Apple tuned in with its
apple-touch-icon.png that some (interestingly non-iOS) clients attempt to load from every single site visited. If your site does not handle those well-known icon URLs, it may result in unnecessary requests and wasted bandwidth. For example, the 404 error document may be much larger than a regular favicon. Many Web sites have developed the habit of sending redirects (to potentially better suited URLs) instead of 404s. In this case, the browser could be redirected to the home page in search for the favicon, resulting in two (or more) requests which may return lots of unnecessary HTML. Our new error handling for
missing-icons does not paint an icon for you, but it replaces all non-icon answers for well-known icon URLs with a tiny error document.
To save even more bytes, more clients including Microsoft Internet Explorer and Mozilla Firefox now qualify to use Zorro images as a replacement for transparent PNG images with photo characteristics.
There are many more improvements and bug fixes in this release. And as usual, you will find all third-party libraries freshened up.
missing-iconsprevents wasted bandwidth for missing icon files (#32592)
style-concatnow works with all CSS files referenced as
font/*are passed compressed using gzip or deflate compression (#26870)
clientTimingis now also bound to the
pplend) events. Additionally, its structure has changed: the
modeparameter was dropped in favour of the new parameters
overfloware now correctly loaded (#31138)
<text-tilter/>config element has been removed. (#32582)
loadAllparameter of the
ac.stage.stage(id,options)method with the optional options parameter, the specific AC-Stage object is now destroyed and reinitialized with the new options. (#32552)
client/hw/typeis no longer read from the DDB export but calculated from the
User-Agentheader field and the
pplnavstartevent is now fired when submitting a form. (#32649)
aelements in AC-Stage elements will no longer get a href attribute if they did not already have one. (#32717)
addElementfunction now works correctly on Windows Phone 8 (#32082)
data-ai-priorityhas been removed. (#32582)
fit.inilimits the HTML/XML parser’s input size to prevent excessive CPU usage when processing large DOMs (#32800)
Content-Lengthfield in the
fit_request.lognow contains the number of originally transferred bytes, even when the response was read from the internal cache (#32521)
fit_request.log, backend request timeouts that occur after the HTTP response headers have been loaded are now logged with HTTP status code
brotli0.4.0 built with
Release date: 2016-07-06
https:// should be considered the norm for Web traffic. It allows for secure communication, the green padlock looks cool, and most important: it enables the efficient and fast data transfers of HTTP/2. However, switching your site to HTTPS may not be trivial. The
force-https feature removes the hassle of changing URLs in your backend. But what can you do about URLs pointing to other domains, like ad scripts, social media or user images? Your browser will show mixed protocol errors and block the loading of these assets. The new
external-media handles this by rewriting foreign
http:// URLs to their secure counterpart. Switch to
Speaking of security, we have improved our transport mapping feature, which is crucial when using the same domain name for Web Accelerator and source, to validate TLS certificates on mapped traffic. E.g. your
example.com certificate is accepted when the requests are routed to the IP resolved for
Besides a lot of bug fixes, this version ships ImageMagick 7 to be up-to-date with the frequent upstream updates, many of which are security related.
external-mediaoption for the
url-rewriting/force-httpssetting to prevent mixed-content warnings (#31318)
$!*'()-_.~/?:@will now remain unencoded in query parameters in backend requests (#30863)
fitdetectioncookie was improved to better cope with clients sending multiple
Image Scalerreturns file path instead of image data, in case of
url(…)in CSS (#30500)
script-managerenabled, an empty resource does not result in an additional request anymore. (#32216)
detectioninitevent can be used to register custom detection code. (#31628)
client/hw/input/touchscreennow always has the value
viewport/orientation-switchablenow always has the value
true. As a result, filtering of media queries with the
orientationmedia feature is not possible anymore. (#32380)
viewport/orientation-switchableis deprecated. (#32380)
Release date: 2016-06-08
This release is a maintenance release comprising mostly bug fixes, security patches and 3rd party updates. However, there are some interesting improvements, too.
But security comes first! The Web Accelerator uses the ImageMagick library for image scaling. Recently, a number of security vulnerabilities were discovered in ImageMagick. With this release, we address these with a
policy.xml as recommended by the vendor, as well as some patches that remove potentially exploitable code that we do not use. This is why we recommend all customers to update to 14.4.1 as soon as possible.
Until now, the cache for scaled or compressed images was valid for one day – regardless of the resource’s actual caching headers. Now these headers are respected, allowing shorter TTLs. However, you may still override this with the interval cache.
With the new scaling option
height attributes of images are set in the HTML document. This allows browsers to render the page quicker, because no reflows are necessary after the images have been downloaded. For pages with delayed images, this avoids “jumping pages” if image dimensions are missing.
Still in Beta, but nevertheless exciting, we provide an optional new URL scheme. With trailing marks enabled, the URL marks needed for URL optimization (such as caching) are moved to the last path segment. In 1:1 setups where a single origin is mounted onto
/, all paths are identical to the origin. In combination with the new
force-transparent switch, this allows for transparent cookies with arbitrary
policy.xmlfor ImageMagick to prevent various vulnerabilities as described by ImageTragick (CVE-2016-3715) (#32116)
url-rewriting/trailing-markschanges the position of URL Marks to the basename of the URL BETA (#32057)
force-transparent(BETA) to be used together with trailing marks (#32057)
heightattributes are present for cached images (#31870)
srcsetrewriting role to rewrite custom attributes like
text/htmlresponses are no longer sniffed for image content (#32131)
viewport/device-pixel-ratioproperty is only determined via detection. (#32209)
register-regexapplies regex replacements after all other content modifications and Adaptive Components (#32103)
pplnavendevent were added to Partial Page Loading to improve usability of the
pplreadyevent, that is triggered immediately when the Partial Page Loading session is initialized, even before the load event (#31809)
popstateevents in PPL after
POSTrequests or redirects will now be processed by PPL (#31953)
formelement parsing on Mobile Safari (on iOS 8) after navigating with PPL (#32129)
fitadmin config getor
fitadmin config listnow return Boolean configuration setting values as
Release date: 2016-05-04
This release brings a performance update to the “rear side”. When communicating with an origin server, a TCP connection needs to be established. This comprises multiple round trips. For encrypted connections, an additional TLS handshake is necessary to setup parameters for secure communication. Depending on the network latency this adds up to a significant amount of waiting time in the critical path. Furthermore, the throughput of a freshly established TCP connection is limited. Only after a number of network packages have been exchanged, the throughput increases.
With 14.4. we start recycling TCP connections to avoid these performance penalties. This reduces connection overhead and improves request performance. The savings are especially high for TLS backends and (very) remote origins. On our production systems we have seen average speed-ups of up to 20%. But even in low latency environments with plain HTTP there are some milliseconds to scrape. Check your
fit_request.log to see what’s in it for you!
Besides that, we have fixed a number of bugs. But in case something goes wrong nevertheless, you can now fail with grace with a custom error page.
Please note that this major release includes deprecations and removals.
In the Beta section we have introduced a page cache for PPL: When navigating back and forth in the browser history, no HTTP request or HTML parsing will be required. Instead, the previous pages’ DOM objects may be reused, resulting in much faster and less interruptive history navigation.
no-storein order to avoid redirect loops on Opera Mini browsers (#31581)
document-write-deferringwill now copy all attributes on scripts created with
request/queryin the Delivery Context (#31191)
image/svg/filterwas renamed to
image/svg/zorro, because the way Zorro images are created is subject to change (#31828)
fit/client/pplis only determined if Partial Page Loading is activated (#31493)
jsis now set to
truefor all clients, if not detected otherwise (#31848)
client/hw/display/default-orientationproperty has been deprecated (#31663)
client/hw/input/touchscreenproperty has been deprecated (#31664)
Release date: 2016-04-06
We have learned a lot from analyzing our image traffic. Therefore, this minor release focusses on bug fixes and enhancements of image processing.
Many images are small enough, in terms of dimensions, to fit into current clients’ viewports. That is because many mobile devices have high density displays (e.g. Apple’s retina display) resulting in lots of pixels. Therefore, Image Scaling and Compression are now separately configurable. This can be useful for CSS documents with image sprites that should not be scaled but still be compressed.
If an image is known to be small enough for the requesting client, FIT will automatically switch from Image Scaling to Image Compression. The latter uses image URLs without sizing parameters, resulting in a better cache hit rate (in FIT and in downstream caches) and reduced disk and CPU usage.
Another finding in our image statistics is that in certain cases image weight (i.e. file size) may be increased by our image processing. Then, the smaller original image will be sent to the client instead of the processed image. We call that
pass-on-enlargement. Since animated GIF images are hard to optimize and are practically never scaled, they are no longer processed at all.
We have further improved SVG as a replacement for transparent PNGs (also known as “ZorroSVG”). The alpha mask is smaller. And a bug showing artifacts at image borders was fixed.
Developers will find some interesting enhancements, too. The
content DC properties for example let you run Flow actions depending on the type of content (e.g.
js). The detection page can now be disabled on a per-URL basis.
metatags directly after the first
falseto revert to the behaviour of FIT before 14.3.0. (#29896)
bodyelements are moved to the implicit
bodyelement created by the HTML5 parser (#30997)
position: fixedelements for delayed images with visibility prioritization. (#31197)
Animated GIFswill no longer be scaled or compressed (#30456)
$!*'()-_.~will now remain unencoded in query parameters in backend requests (#30863)
Authorizationheader in the request to the source server (#31031)
Release date: 2016-03-09
This update brings a significant performance boost to the server-side! The overall CPU usage is reduced by 25% on average. Depending on the features in use, sites may see even larger savings in CPU time. This is not only a benefit in terms of hardware savings. Since the responses are generated more quickly, resulting in shorter times-to-first-byte, it can be a major improvement to your site’s critical render path. These performance gains result from the update of the underlying PHP engine to the much acclaimed version PHP7.
The Web Accelerator was improved, too. As images are a very important part of the user experience, support for WebP and SVG images is now determined automatically by the client feature detection. Before 14.3, these capabilities were taken from our Client Description Repository. With the new on-the-fly detection we have more accurate information when dealing with new, unknown clients. FIT now uses URL encoding instead of base64 when Inlining SVG images, which results in more efficient transfer compression and a reduced payload for the client.
The documentation now has a section on beta features. It offers an overview of experimental features that we make available for pre-production use. Any developer feedback on these features is highly appreciated!
With this major release, we have also removed some formerly deprecated features. Please note that Session Management and the Load Brake have been removed.
base64, thereby reducing the transfer size (#30398)
noscriptelements are now removed in PPL requests (#30844)
<head>element was misspelled (#30947)
metatags where a
charsetattribute already exists (#30769)
'use strict'directive (#30673)
ai-fragmenttags are now stripped from the output document, to prevent interference with CSS relying on their absence (e.g. child selectors) (#31086)
theadelements in combination with HTML minifying. (#30284)
quality:-1now works properly with the image url API (#31078)
ai-rewriteimagenow respects the
image-scalingsettings defined in
util.jssnippet, resulting in a wrong preview title (#30864)
errorlogis now less verbose to be better “tailable” (#30803)
urlencode()does not encode the
~(tilde) character anymore (#30796)
md5()no longer implicitly converts the UTF-8 input to ISO-8859-1. This may lead to different result hashes! (#30796)
-webkit-overflow-scrolling:touchis used in AC-Scroll (#28462)
FIT_JOB_HOST_URLhas been renamed to
FIT_STATUS_URL. It is only used for
apachectl/phpfpmctl (full)status. (#30415)
Release date: 2016-02-10
The first release of 2016, FIT 14.2.0 primariliy brings a set of new features regarding image optimization.
FIT now also uses 16-bit algorithms for internal image calculation and a better gamma correction. Better handling for truecolor PNGs that are transcoded to WebP format was implemented and developers now may use
ai-scaling-width/height values that will scale images bigger than the target viewport. Additionally bringing new scroll speed dependent loading of visibility delayed images and options to control the viewport offset for loading these, FIT 14.2.0 arguably offers the best automated image optimization we ever had.
For assuring high quality in development and maintenance of projects we introduced an XPath function and a
fit.ini setting. The
watch() XPath function allows you to watch XPath expressions and in case of errors will create entries in the
fit_alert.log. In combination with the new
FIT_DEPRECATION_EXCEPTION, which will cause exceptions to be thrown when deprecated features are used, you can minimize the chance that your projects will run into future issues.
We also did a rework of our user agent detection. FIT now uses the common UA Parser project for client matching. It provides a robust set of regular expressions that are less sensitive to the frequent version updates (of both operating systems and browsers) seen in user agent strings. However, we still maintain our own Client Description Repository that stores client capabilities, features and properties. After identifying the client, our CDR data is accessed to build up the Delivery Context.
Updates of third party software and bugfixes complete this release.
-1in the created URL (#30267)
loadevent of delayed images is now dependent on the current scroll speed (#30198)
jsoncontent types (e.g.
HEADrequests and replies with various non-200 HTTP status codes when the
parseaction performs image scaling (#30300)
rel="stylesheet"and the same
idattributes of script elements (#30377)
Content-Typeheader is set accordingly (#30507)
srcattribute containing a data URI and a
visibilityprioritization would cause FIT to load the erroneous image multiple times (#30701)
styleelements that contained
FIT_DEPRECATION_EXCEPTIONin fit.ini – this is activated by default in the SDK (#30099)
semin Apache configuration to avoid restart errors on CentOS 7.2 (#30391)