The dev team was working on a bug fix during the last two weeks in close collaboration with Stéphane, the main developer of the original Freestyle program. The problem was that the view map calculation did not take mesh face smoothness into account.
So far, mesh geometry data was treated as if all faces were flat (i.e., face smoothness was just ignored), which implied that the view map was composed of jagged feature edges as illustrated in the example below. The left image is a rendering result of a torus mesh, where strokes are drawn along visible and hidden silhouette lines of the torus. The right image shows the feature edges (highlighted) from a three-dimensional non-camera viewpoint. These jagged polylines are what constitute the strokes that appear in the render. As you see, the feature edges exactly lie on edges in the mesh geometry data. This causes somehow jaggy strokes in the rendering result (especially in the part of the render where strokes make a quick turn).
When the face smoothness is taken into account in the view map calculation, the resulting feature edges totally differ from the outcome shown above. As seen in the example below, the three-dimensional feature edges representing the silhouette lines are much smoother. The left image is a render of the same torus mesh, whereas the right image shows the smooth feature edges resulting from the view map calculated by accounting for the smoothness property of mesh faces. Note that the smooth feature edges are no longer constrained by the original mesh topology.
As far as the dev team has experimented with several test scenes, taking account of mesh face smoothness results in a cleaner stroke topology. The downside is that the visibility of smooth silhouette lines is mathematically not well-defined. This means that rendering results may suffer from some artifact due to incorrect line visibility. To our knowledge, the visibility of smooth silhouettes is still an open problem in computer graphics research. Therefore, we decided to introduce an option for enabling/disabling mesh face smoothness in the view map calculation. In the Freestyle branch revision 33350 and later, you have the new option “Face Smoothness” in both the Parameter Editor mode and Python Scripting mode. When the option is enabled, the view map calculation accounts for face smoothness. This option is intended to allow users to try the two ways of view map calculation with and without face smoothness and find which one would fit individual application needs. For now the option is disabled by default in order to avoid the aforementioned visibility issue.
Freestyle integration into Blender 