Jeff Muizelaar

Moving patches between git and hg

2011-10-19T13:17:00.000-07:00

Moving patches between git and hg is currently not very easy. I found script that converts in one direction and I added script that goes in the other direction. The scripts are available here: https://github.com/jrmuizel/patch-converter. Hopefully, this will make it a bit easier.

WebGL considered harmful?

2011-06-16T14:04:00.000-07:00

Today Microsoft posted an article titled "WebGL considered harmful". It seems like a lot of their arguments against WebGL also apply to Silverlight 5's XNA 3D graphics support. It, like WebGL, allows authors to write shaders using HLSL. I wonder, if you reframe their article by replacing WebGL with Silverlight 5, is anything untrue? If so, how does Microsoft solve these problems?

Silverlight XNA 3D considered harmful
Microsoft's Silverlight 5 XNA 3D technology is a low-level 3D graphics API for the web.

One of the functions of MSRC Engineering is to analyze various technologies in order to understand how they can potentially affect Microsoft products and customers. As part of this charter, we recently took a look at XNA 3D. Our analysis has led us to conclude that Microsoft products supporting XNA 3D would have difficulty passing Microsoft’s Security Development Lifecycle requirements. Some key concerns include:
Browser support for Silverlight 5 directly exposes hardware functionality to the web in a way that we consider to be overly permissive
The security of Silverlight 5 as a whole depends on lower levels of the system, including OEM drivers, upholding security guarantees they never really need to worry about before. Attacks that may have previously resulted only in local elevation of privilege may now result in remote compromise. While it may be possible to mitigate these risks to some extent, the large attack surface exposed by Silverlight 5 remains a concern. We expect to see bugs that exist only on certain platforms or with certain video cards, potentially facilitating targeted attacks.

Browser support for Silverlight 5 security servicing responsibility relies too heavily on third parties to secure the web experience
As Silverlight 5 vulnerabilities are uncovered, they will not always manifest in the Silverlight 5 API itself. The problems may exist in the various OEM and system components delivered by IHV’s. While it has been suggested that Silverlight 5 implementations may block the use of affected hardware configurations, this strategy does not seem to have been successfully put into use to address existing vulnerabilities. It is our belief that as configurations are blocked, increasing levels of customer disruption may occur. Without an efficient security servicing model for video card drivers (eg: Windows Update), users may either choose to override the protection in order to use Silverlight 5 on their hardware, or remain insecure if a vulnerable configuration is not properly disabled. Users are not accustomed to ensuring they are up-to-date on the latest graphics card drivers, as would be required for them to have a secure web experience. In some cases where OEM graphics products are included with PCs, retail drivers are blocked from installing. OEMs often only update their drivers once per year, a reality that is just not compatible with the needs of a security update process.

Problematic system DoS scenarios
Modern operating systems and graphics infrastructure were never designed to fully defend against attacker-supplied shaders and geometry. Although mitigations such as Direct3D 10 may help, they have not proven themselves capable of comprehensively addressing the DoS threat. While traditionally client-side DoS is not a high severity threat, if this problem is not addressed holistically it will be possible for any web site to freeze or reboot systems at will. This is an issue for some important usage scenarios such as in critical infrastructure.

We believe that Silverlight 5 will likely become an ongoing source of hard-to-fix vulnerabilities. In its current form, XNA 3D in Silverlight 5 is not a technology Microsoft can endorse from a security perspective.

We recognize the need to provide solutions in this space however it is our goal that all such solutions are secure by design, secure by default, and secure in deployment.

The problems Microsoft is worried about are real, and they don't have any easy solutions. At the same, I don't think we need to wait for perfect answers before trying. With Silverlight 5's 3D support, it looks like Microsoft feels the same way.

WebP

2011-04-20T07:26:00.000-07:00

Overall the reception to WebP that I've seen so far has been pretty negative. Jason Garrett-Glaser wrote a popular review, but there have been similar response from others like Charles Bloom. Since these reviews, the WebP encoder has improved on the example used by Jason (old vs. new) but it's still not a lot better than a decent JPEG encoding. I also have a couple of thoughts on the format that I'd like to share.

Google claims it's better than JPEG but this study has some problems and as a result, isn't very convincing (Update: Google has a new study that's better). First, they recompress existing JPEG's. This is unconventional. Perhaps recompressing JPEG's is their target market, but I find that a little weird and it should at least be explained in the study. Second, they use PSNR as a comparison metric. This is even more confusing. PSNR has, for a while now, been accepted as a poor measure of visual quality and I can't understand why Google continues to use it. I think it would help the format's credibility if Google did a study that used uncompressed source images, SSIM as a metric and provided enough information about the methodology so that others could reproduce their results.

WebP also comes across as half-baked. Currently, it only supports a subset of the features that JPEG has. It lacks support for any color representation other than 4:2:0 YCrCb. JPEG supports 4:4:4 as well as other color representations like CMYK. WebP also seems to lack support for EXIF data and ICC color profiles, both of which have be come quite important for photography. Further, it has yet to include any features missing from JPEG like alpha channel support. These features can still be added, but the longer they remain unspecified, the more difficult it will be to adopt.

JPEG XR provides a good example of what features you'd want from a replacement for JPEG. It has support for an alpha channel and HDR among others. Microsoft has also put in the effort to have it formally standardized. However, it too is not without problems. The compression improvements it claims haven't matched evaluations other parties have done. I don't know enough about JPEG XR to say whether this is because the encoders are bad or because the format is not really that great.

Every image format that becomes “part of the Web platform” exacts a cost for all time: all clients have to support that format forever, and there's also a cost for authors having to choose which format is best for them. This cost is no less for WebP than any other format because progressive decoding requires using a separate library instead of reusing the existing WebM decoder. This gives additional security risk but also eliminates much of the benefit of having bitstream compatibility with WebM. It makes me wonder, why not just change the bitstream so that it's more suitable for a still image codec? Given every format has a cost, if we're going to have a new image format for the Web we really need to make it the best we can make it with today's (royalty-free) technology.

Where does that leave us? WebP gives a subset of JPEG's functionality with more modern compression techniques and no additional IP risk to those already shipping WebM. I'm really not sure it's worth adding a new image format for that. Even if WebP was a clear winner in compression, large image hosts don't seem to care that much about image size. Flickr compresses their images at libjpeg quality of 96 and Facebook at 85: both quite a bit higher than the recommended 75 for “very good quality”. Neither of them optimize the huffman tables, which gives a lossless 4–7% improvement in size. Further, switching to progressive JPEG gives an even larger improvement of 8–20%.

History has shown that adoption of image formats on the internet is slow. JPEG 2000 has mostly failed on the internet. PNG took a very long time, despite having large advantages. I expect that adoption may even be slower now than it was in the past, because there is no driving force. I would also be surprised if Microsoft adopted WebP because of their stance on WebM and their involvment in JPEG XR. Can WebP succeed without being adopted by all of the major web browsers? It's hard to say, but it wouldn't be easy. Personally, I'd rather the effort being spent on WebP be spent on a improved JPEG encoder or even an improved JPEG XR encoder.

Is JPEG still great? No. Is there a great replacement for it? It doesn't feel like we're there yet.

Drawing Sprites: Minimizing draw calls

2011-03-02T21:07:00.000-08:00

One reason OpenGL is so fast is that it allows applications to provide large chunks of work to be done in parallel. When drawing sprites with WebGL, it's important to make an effort to take advantage of this by minimizing the number of draw calls. This is true with OpenGL, but even more so with WebGL because each draw call requires extra validation.

Unfortunately, minimizing draw calls isn't always easy. It's often impractical or impossible to draw all your geometry at once because the geometry must share the same texture(s). FishIE used a single sprite from the beginning, which made it easy to draw everything at once. If possible, move as many sprites into the same texture as possible and sort or group sprites using the same texture into a single draw call. It may also be possible to use multi-texturing, but depending on the GPU architecture, this can cause all textures to be read for each sprite which will have dramatic impact on performance because of limitations on texture bandwidth.

The performance difference between drawing sprites individually versus all at once can be pretty big. I made another version of the FishIE demo that draws each sprite individually. This version draws 2000 fish at 10fps on my test system, while the original WebGL FishIE can do 4000 fish at 60fps on the same system. Since the same texture is used for all sprites I did not have to rebind the texture for each sprite; doing so would likely decrease performance further.

Designing an application around these limitation can be tricky, but often the application is in a better position to make compromises or take short cuts than a more general Canvas 2D implementation would be.

Drawing Sprites: Canvas 2D vs. WebGL

2011-02-28T13:29:00.000-08:00

Lately I've seen a lot of graphics benchmarks that basically just test image blitting/sprite performance. These include Flying Images, FishIE, Speed Reading and JSGameBench(Update: I just saw the blog post for the WebGL JSGameBench. This further confirms my claim that WebGL is a better way to do sprites). They all try to draw a bunch of images in a short amount of time. They mostly use two techniques: positioned images or canvas' drawImage. Neither of these methods is particularly well suited to this task. Positioned images have typically been used for document layout and the Canvas 2D API was designed as a JavaScript binding to CoreGraphics which owes most of its design to Postscript. Neither were designed for high performance interactive graphics. However, OpenGL, and its web counterpart WebGL, was designed for exactly this.

To show off some of the potential performance difference available, I ported the FishIE benchmark to WebGL. Along the way I discovered some different problems and ways to solve them.

The problem, once the overhead of Canvas 2D is removed, is that FishIE very quickly becomes texture read bound. I noticed that the FishIE sprites have a lot of horizontal padding. This padding was included in the drawImage calls which causes us to do a bunch of texture reads for transparent pixels. Trimming this down a little gave a noticeable framerate boost.

An even bigger cause of texture bandwidth waste is that the demo uses a large sprite to draw a small fish. Fortunately, OpenGL has a great solution to this problem: mipmaps. Mipmaps let the GPU use smaller textures when drawing smaller fish, which can dramatically reduce the texture bandwidth required. They also improve the quality of small fish by eliminating the aliasing that occurs when downscaling by large amounts.

Mipmapping is a good example of the flexibility that WebGL allows. Canvas 2D aims to be an easy to use API for drawing pictures, but this ease of use comes at some cost. First, the Canvas 2D implementation has to guess the intents of the author. For example drawImage on OS X does a high quality lanczos down scaling of the image. Direct2D just does a quick bilinear down scale. This makes it difficult for authors to know how fast drawImage will be. Further, because the design of Canvas 2D is inspired by an API for describing print jobs, it's not well suited to reusing data between paints.

Try out the difference with these two modified versions of FishIE:

The method I used to port FishIE to WebGL is pretty straight forward so I expect that any of the other benchmarks listed above could also be easily ported to WebGL.

Pushing the limits

Once the number of fish becomes high enough we run into Javascript performance problems. FishIE has some Javascript problems that make things worse than they need to be. First, it loops over the fish with "for (var fishie in fish) {". This can end up using 10% of the total CPU time. The problem with this code is that converts all of the array indices to strings and then uses those strings to index into the array. It also has the problem that any additional properties added to the array will also show up as index values, which is likely not the intent of the author.

Second, each fish object includes a swim() method. Unfortunately, in the FishIE source swim() is a closure inside the Fish() object. This means that the swim() method is different for each Fish which makes things worse for Javascript engines.

Fixing both of these problems, and making the fish really small lets us get an idea of how many sprites we can actually push around. Here's a final version. If I disable the method jit (bug 637878) and run at an even window size (bug 637894) I can do 60000 fish at 30fps, which I think is pretty impressive compared to the 1000 that the original Microsoft demo does.

Updated mozilla-cvs-history git repo

2011-02-18T12:55:00.000-08:00

I recently ran git gc --agressive on the cvs history git repository mentioned here. It's now 543M, down from 986M. I've also uploaded a copy to github.

Clone timings

2011-02-10T10:43:00.000-08:00

Chris Atlee was wondering how clone times differ between git and mercurial so I ran a quick test on a fast linux machine.

$ time git clone git://github.com/doublec/mozilla-central.git
real 1m33.478s

$ time git clone mozilla-central moz2
real 0m2.559s

$ time hg clone http://hg.mozilla.org/mozilla-central/
real 3m22.510s

$ time hg clone mozilla-central moz2
real 0m20.660s

historical mozilla-central git repository

2011-01-12T16:37:00.000-08:00

A number of people use git to work with the mozilla hg tree. In the past I've wanted the entire history as a git repo so I converted the old CVS repository to git and put it up on people.mozilla.org.

You can set it up as follows:


git clone http://people.mozilla.org/~jmuizelaar/mozilla-cvs-history.git
git clone git://bluishcoder.co.nz/git/mozilla-central.git

cd mozilla-central/.git/objects/pack
# set up symbol links to cvs-history pack files
ln -s ../../../../mozilla-cvs-history/.git/objects/pack/pack-5b5d604ab48cf7bc2a6b4495292fa8700a987c5f.pack .
ln -s ../../../../mozilla-cvs-history/.git/objects/pack/pack-5b5d604ab48cf7bc2a6b4495292fa8700a987c5f.idx .
cd ../../

# add a graft from the last revision in the mozilla-central repo
# to the first revision in the cvs-history
echo 2514a423aca5d1273a842918589e44038d046a51 3229d5d8b7f8376cfb7936e7be810635a14a486b > info/grafts

Now you have a git repository containing all of the history. You can update the mozilla-central repository as you normally would. The conversion isn't perfect, but it's been good enough to have working blame back into cvs time.

Firefox acceleration prefs changing

2011-01-11T14:05:00.000-08:00

I just landed a changeset that changes the names of the layer acceleration prefs in Firefox.

The old prefs were:
layers.accelerate-all
layers.accelerate-none

The new prefs are:
layers.acceleration.disabled
layers.acceleration.force-enabled

layers.accelerate-all previously defaulted to 'true' on Windows and OS X. Which meant that there was no easy way to force layer acceleration on if your card had been blacklisted for some reason. The new prefs allow the blacklist to be overwritten. The old prefs are not being migrated over to the new names. If you have a problem with the defaults, please file bugs.

Trying out AVX

2011-01-08T18:42:00.001-08:00

Intel's new Sandy Bridge CPUs came out this week and they support a new set of instructions called AVX. The AVX instructions are a much bigger change than the usual SSE revisions in the past few micro-architectures. First of all, they double the 128 bit SSE registers to 256 bits. Second, they introduce an entirely new instruction encoding. The new encoding switches from 2 operand instructions to 3 operand instructions allowing the destination register to be different than the source registers. For example:
  addps r0, r1       # (r0 = r0 + r1)
         vs.
  vaddps r0, r1, r2  # (r0 = r1 + r2)
This new encoding is not only used for the new 256 bit instructions, but also for the 128 bit AVX versions of all the old SSE instructions. This means that existing SSE code can improved without requiring a switch to 256 bit registers. Finally, AVX introduces some new data movement instructions, which should help improve code efficiency.

I decided to see what kind of performance difference using AVX could make in qcms with minimal effort. If you use SSE compiler intrinsics, like qcms does, switching to AVX is very easy; simply recompile with -mavx. In addition to using -mavx, I also took advantage of some of the new data movement instructions by replacing the following:

  vec_r = _mm_load_ss(r);
  vec_r = _mm_shuffle_ps(vec_r, vec_r, 0);

with the the new vbroadcastss instruction:
vec_r = _mm_broadcast(r);
Overall, this change reduces the inner loop by 3 instructions.

The performance results were positive, but not what I expected. Here's what the timings were:

SSE2:	75798 usecs
AVX (-mavx):	69687 usecs
AVX w/ vbroadcastss:	72917 usecs

Switching to the AVX encoding improves performance by more than I expected: nearly 10%. But adding the new vbroadcastss instruction, in addition to the AVX encoding, not only doesn't help, but actually makes things worse. I tried analyzing the code with the Intel Architecture Code Analyzer, but the analyzer also thought that using vbroadcastss should be faster. If anyone has any ideas why vbroadcastss would be slower, I'd love to hear them.

Despite this weird performance problem, AVX seems like a good step forward and should provide good opportunities for improving performance beyond what's possible with SSE. For more information, check out this presentation which gives a good overview of how to take advantage AVX.

Improved Hardware Acceleration in Fennec

2010-12-18T21:53:00.000-08:00

On Thursday night, after the all-hands party, Matt Woodrow landed a beautiful refactoring of our texture upload code. This should give a noticeable improvement in scrolling performance when accelerated layers are enabled and hopefully fixes some of the problems people were seeing there. It also improves texture upload performance on OS X.

Unfortunately, there are still two bugs that are keeping us from enabling accelerated layers by default:
Bug 619615 - Hangs on Nexus One
Bug 619539 - Startup crashes on Droid

Any help debugging these problems would be greatly appreciated.

Hardware Acceleration on Fennec

2010-12-15T12:24:00.000-08:00

It's now possible with current nightlies to use OpenGL for compositing in Fennec on Android. To turn it on, go to about:config and set "layers.accelerate-all" to "true" and restart. If it's working you can go to about:support and the Graphics section will say "1/1 OpenGL".

It would be great if people can test it and let me know how it goes.

Dealing with mach_kernel in Shark

2010-11-08T11:54:00.000-08:00

Sometimes when profiling a bunch of time ends up in mach_kernel. Figuring out why isn't always easy but here are two tips that should help a bit:

You can get better symbols for mach_kernel by downloading a KernelDebugKit
This can help a bit when trying to figure out what's happening in the kernel. For example, _dtrace_get_cpu_int_stack_t becomes _mach_call_munger.

Shark has a System Trace profiling mode. This can show you what code is causing the kernel to do work. It can break down time by system call or by vm fault which should account for most things.

While trying this out I noticed we were spending a fair amount of time in ChildViewMouseTracker::WindowForEvent(NSEvent*). This gave me the idea that the reason that Firefox causes the WindowServer process to start using a huge amount of CPU is because we tell the WindowServer to give us all of the mouse events instead of the ones only targeted at our window. Presumably this causes the WindowServer to build up a very large queue of events when the Firefox process is stopped and thus use lots of CPU. This turns out to be the case. nsToolkit::RegisterForAllProcessMouseEvents causes us to listen to all mouse events and disabling the code there fixes the problem. Bug 611068 tracks the problem.

Reviewing in vim

2010-05-28T14:22:00.000-07:00

Bugzilla's review interface is poor. I find a mild improvement is possible by copying the review text into an editor and reviewing it there. One of things that makes this experience better is syntax highlighting. Here's a modification of vim's diff highlighting script that works with quoted patches. Adding the following to one's .vimrc will get it used for .review files:


au BufNewFile,BufRead *.review                  setf review

Reproducing bugs on complicated webpages

2010-01-22T14:52:00.000-08:00

A while back, I was running into hangs on Facebook with the html5 parser. I wanted to reproduce the problem locally and try to minimize it. However, Facebook is pretty complicated and loads content using ajax, so the traditional approach of saving the page didn't work. I needed a different approach, so I hacked up a http proxy to be able to record and replay server interactions. It works by writing out the result of each GET request to a file named after the Request-URI. After you're done recording an interaction with the server, you can switch to replay mode and instead of proxying, the recording will be played back to the browser.

I should mention a couple of limitations to this approach. First, it assumes that the resources specified at particular urls don't change over the recording session. Second, it assumes that the resources requested are dependent only on the data recorded and not on the time or anything like that. Even with these limitations, the proxy seems to work well enough.

It's currently pretty hacky, but you can grab it from git://anongit.freedesktop.org/~jrmuizel/http-recording-proxy. There's a README in the repository explaining how to run and replay. I'd love to hear if this of any use to anyone else or if you have patches for improving it.

Graphics performance in Firefox 3.6

2010-01-22T07:47:00.000-08:00

One of the performance improvements included in Firefox 3.6 is a new path rasterizer for use on Windows. This new rasterizer improves vector graphics performance substantially.

The previous rasterizer is designed for a XRender trapezoid model of rasterization. In this model, to fill a polygon, we first tessellate it into a collection of trapezoids. Next, each of these trapezoids is rasterized and the result added to mask image. Finally, this mask is used to composite the filled polygon. This design can work well if we rasterize multiple trapezoids in parallel as could be done on a GPU. However, when we're using the CPU to sequentially rasterize and blend each trapezoid it's not the most efficient approach.

Scanline rasterization is the textbook method for filling polygons and when using a CPU it can be more efficient than tessellating. Instead of breaking the polygon into trapezoids we iterate over each scanline of the mask image. For each scanline, we iterate over the edges that it intersects and fill the pixels in-between the edges to produce the mask image.

M Joonas Pihlaja contributed a scanline rasterizer to cairo as part of a Google Summer of Code project and it's included in Firefox 3.6. This new rasterizer makes a pretty significant difference when filling complex paths. For example, this test draws a spinning map of the world using canvas. In Firefox 3.5, I get about 6-7fps. Using Firefox 3.6, it's nearly 4x faster with about 19-24fps.

Geometry Homework

2009-10-07T21:46:00.000-07:00

Drawing arcs is a common task in computer graphics. Arcs are typically drawn using an approximation by set of cubic Bezier splines. These splines are passed through the regular rendering pipeline where they are usually subdivided into a set of line segments and then rasterized. A circle, for example, can be represented by four cubic splines: one for each 90 degree arc.

An arc of up to 90 degrees can be approximated by a single cubic Bezier spline reasonably well. To find an appropriate approximating spline we set the start and end points of the spline to match the arc and then can find the middle two points using the following formula, where A is the start angle and B is the end angle:

h = 4/3 * tan ((B - A)/4)

pt1 = {center.x + r*cos(A), center.y + r*sin(A)}
pt2 = {center.x + r*cos(A) - h*r*sin(A), center.y + r*sin(A) + h*r*cos(A)}
pt3 = {center.x + r*cos(A) + h*r*sin(B), center.y + r*sin(A) - h*r*cos(B)}
pt4 = {center.x + r*cos(B), center.y + r*sin(B)}

This approach works quite well and is currently used by cairo. However, the problem with this approach is that it requires converting to polar coordinates and then back to Cartesian ones. This conversion is slow because it requires the use of trigonometric functions.

Fortunately, this paper gives a solution that avoids the conversion to polar coordinates:

ax = pt1.x - center.x
ay = pt1.y - center.y
bx = pt4.x - center.x
by = pt4.y - center.y

q1 = ax*ax + ay*ay
q2 = q1 + ax*bx + ay*by

k2 = 4/3 (sqrt(2*q1*q2) - q2) / (ax*by - ay*bx)

pt2.x = pt1.x - k2*ay
pt2.y = pt1.y + k2*ax
pt3.x = pt4.x + k2*bx
pt3.y = pt4.y - k2*by

Unfortunately, there is no explanation, derivation or proof for this formula. Even more problematic, the formula for k2 becomes less stable as pt1 approaches pt4, becoming NaN when pt1 equals pt4.

Therefore, the homework problem has two parts:

1. Give an explanation or derivation for the formula for k2 provided above.
2. Provide a similar formulation for pt2 and pt3 that doesn't degenerate as pt1 and pt4 become close or prove that one doesn't exist.

The best answer will be credited in the new cairo stroking code.

qcms — now faster

2009-10-02T12:51:00.000-07:00

Thanks to some optimization work by Steve Snyder, qcms is even faster than before.

What follows is a chart with the new performance numbers:The benchmark is the same as last time but run on a slightly slower computer using OS X v10.6 instead of 10.5. As the chart shows, the new qcms code is more than twice as fast as the previous code. In addition to the performance improvement, the new code includes a version that only uses SSE1 instructions. This will be especially helpful for those with older computers where the time spent doing color correction isn't as negligible as it is on faster computers.

When running this benchmark again, I noticed that the performance of lcms had drastically improved since the last time I had run the benchmark. Why was lcms so much faster on 10.6? What had changed? The default architecture target: in OS X 10.6, the compiler builds 64 bit binaries by default. Still, it was surprising that compiling for 64 bit could nearly double the performance.

The large difference, it turns out, can largely be attributed to the MAT3evalW¹ function. This function multiplies a 1×3 matrix with a 3×3 one using 9 32×32→64 multiplications. GCC can usually optimize these multiplications by using the 32×32→64 multiply instructions, however that wasn't happening in 32 bit mode. Instead of the expected 9 multiplies, we get 18 multiplies and a bunch of housekeeping work, likely caused by the 64 bit additions and additional register pressure. In 64 bit mode, however, we get the code that you'd expect. This only takes 38 instructions versus the 169 instructions the 32 bit build uses. With a difference like that in the inner loop, it's easy to see why the 64 bit build is so much faster.

1. MAT3evalW has a handwritten assembly version that should be faster than the one that GCC generates, unfortunately it is MSVC only.

qcms - color management for the web

2009-06-02T08:57:00.000-07:00

qcms is a brand-new color management system replacing Mozilla's previous solution, lcms. The browser's color management system allows us to use data in photos on the web and data about your monitor to make sure that colors look the same everywhere. This helps photographers and site designers control the appearance of their images more precisely, and makes the web a prettier place.

Normally, rewriting a large chunk of code from scratch is a bad idea. However, we were in a situation that made rewriting pretty attractive. First, we were only using a small portion of lcms's functionality, but we were paying for the large code base in both maintainability and threat surface. Second, we were already maintaining substantial modifications which was an additional maintenance burden. Finally, our requirements for a color management library are different then the goals of lcms: we need something fast and secure, while lcms seems more focused on functionality, completeness and correctness.

What's new?

qcms is made up of two main parts: the ICC profile parser and the transformation engine. The transformation engine reuses a lot of code from lcms. The ICC profile parser is completely new and written with security and robustness in mind.

The new parser has some key design differences compared to lcms. One of these is the I/O model. lcms has an I/O abstraction layer that abstracts reading from memory and reading from a file. It uses this layer to read what it needs from the profile as it's needed. This means that the full file or memory copy of the profile needs to be kept around for the lifetime of the profile object.¹

qcms uses a simpler model. The entire profile is read into memory and then parsed to construct a profile object. We only keep the information that's needed to construct a transformation between color spaces. After parsing we can close the file or free the memory used to store the profile. Furthermore, since we only parse in-memory, we also don't have to deal with any possible read errors or other file I/O problems during parsing.

Error handling during parsing can be tricky and ridden with security holes. To help deal with this problem, qcms adopts an error handling strategy similar to cairo. Instead of trying to deal with the error immediately and returning an error result up the call stack, we often set a flag to note the brokenness, putting us into an error state, and continue on. To continue successfully, all of the following operations must be completable even while in an error state. However, ensuring this is usually easy, especially if the results will be discarded. When we do get to a place where it's convenient to return, we do so. The big advantage to this approach is that it keeps the error state control flow as similar to the normal control flow as possible. This makes the code easier to read, easier to reason about, and easier to test.

Speed

Last summer, Bobby Holley did a bunch of work to make lcms faster. I was able to reuse that work in qcms. The result is that qcms is one of the fastest color management systems around. Here's a simple test that transforms all the possible RGB components from one RGB colorspace to another. It compares lcms, qcms and ColorSync, the system color management system on OS X.

The tests are here and here.

Current Limitations

qcms currently only supports transformations to and from RGB colorspaces. This covers the vast majority of uses on web; however, it means there's no support for CMYK and many additional profile types. If you are interested in making this code better, let me know!

Conclusion

In the end, we have a library about a tenth of the size and 5 times as fast than what we had previously. I've put the code up at git://git.freedesktop.org/~jrmuizel/qcms. It's portable C and licensed under the same license as lcms so it's pretty easy to change from one api to another. Hopefully other projects can find it useful. Finally, a big thanks to Marti Maria for lcms, without which qcms wouldn't been possible.