Weekly updates December 6-26

The last three weeks were mainly spent for bug fixes and consolidation of the Freestyle renderer.

• A bug in stroke rendering with round/square caps was fixed.  A freeze of the program was caused by a stroke having two vertices that reside (exactly or almost) at  the same position in the 3D space.
• The Parameter Editor mode was improved so that it accepts quote symbols within the names of scene render layers and linesets.
• Incorrect vertex normals in the case of smooth surfaces were fixed.  Many thanks to Stéphane for his careful code review and bug report!
• Compilation errors with CMake on Linux were fixed by a patch from Écrivain through the BA Freestyle thread.  Thanks!

In addition, the Freestyle Python API has been slightly extended by wrappers of the following Operators class methods in the C++ layer:

• getViewEdgeFromIndex(index): Returns a ViewEdge object in the current set of ViewEdges (0 <= index < Operators.getViewEdgesSize()).
• getChainFromIndex(index): Returns a Chain object in the current set of Chains (0 <= index < Operators.getChainsSize()).
• getStrokeFromIndex(index): Returns a Stroke object in the current set of Strokes (0 <= index < Operators.getStrokesSize()).

When you want to iterate over all objects in the current set of ViewEdges, Chains, or Strokes (for instance, in order to find the minimum and maximum Z depths, or sum up the lengths of all strokes), these API functions give a procedural programming way of accessing the objects, instead of relying on implicit side-effects of declarative rules (such as predicates and shaders).

Weekly update November 29-December 5

Since the last blog update, the dev team was working on bug fixes, addition of new features, and consolidation of existing functionalities.

A compilation issue with MinGW32 was fixed thanks to a report from Dazzle, and a bug in Distance from Object modifiers due to a recent change in the trunk about matrix-vector multiplication was fixed based on a .blend file provided by Jonathan Hudson.  We really appreciate the user feedback, which helps the dev team to improve the software.

In the meantime, the Parameter Editor mode was extended with two new feature edge selection options.  One is the “Selection by Group” option and the other is the “Selection by Image Border” option, both in part of linesets in the Parameter Editor mode, as shown in the screenshot below.

The Selection by Group option allows you to select feature edges based on a group of objects (defined through the Groups tab in the Object buttons).  For each lineset, you can specify one group of objects.  The group-based feature edge selection is either inclusive or exclusive.  If inclusive, feature edges are selected when they belong to some object in the group; otherwise, those feature edges not belonging to any object in the group are selected.  When no group is specified, feature edges are selected from all objects in object layers enabled in a render layer.  In the past, render layers were the only way to apply different line styles to different objects in the same scene.  This required the use of the compositor and thus a more and less complicated workflow.  The group-based feature edge selection is intended to address this difficulty, making object-specific line stylization much easier.  The image below is a result using this new feature.  A group of objects comprising a lily in the middle was defined, based on which white lines were drawn.  A different line style was applied to lines of two lilies in the left and right using a separate object group.

The Selection by Image Border option is used to select feature edges within the border of the image being rendered.  The border is defined either by the frame size or a border region (specified by the Shift-B key in a 3D View window).  When you work on a big scene with a lot of objects outside the viewing frustum, enabling this option leads to less memory consumption and possibly shorter rendering time.

Finally, the data flow within the Freestyle renderer was revised with the aim of reducing the amount of memory consumption.  Rendering of a 3D scene in Freestyle involves several data structures holding different representations of the scene and strokes generated from the scene.  The data processing in Freestyle is summarized as follows:

1. First of all, mesh data are imported from the 3D scene and stored in a tree-based data structure.
2. The mesh data are transformed into a data structure called winged edges and stored in a 3D grid structure for efficient spatial search.
3. The winged edges are used for view map calculation, and a separate view map data structure is constructed.
4. The view map is fed to stroke generation by means of style modules (it is noted that the Parameter Editor mode is implemented as a built-in style module).
5. Generated strokes are converted into Blender mesh data for stroke rendering.

Previously, all the data structures were kept until the end of the entire rendering process.  This makes sense in the original, stand-alone Freestyle program, in which the view map, for instance, can be reused for applying different style modules one after another.  This is not the case in Freestyle for Blender, where the view map is recalculated for each render.  Therefore, the memory resources allocated for the data structures should be released soon after they become unnecessary even in the middle of the rendering process.  For this reason, memory usage improvements were made with regard to the aforementioned data structures.  The improvements were verified by a test scene (generated using a modified version of “Arc Sphere.es” in Structure Synth) consisting of 34968 vertices and 52452 faces.  The left image below is a rendering result.  Using a 64-bit Windows machine, memory consumption in the latest revision 33485 was compared with that in revision 33434 without the memory-related updates.  The right image below shows the amount of allocated memory (in the vertical axis) throughout the rendering time (in the horizontal axis).  The maximum amount of allocated memory in the older revision was about 2.138 GB, which reduced to 1.439 GB through the memory usage optimization as well as the Selection by Image Border option.

 

Experiments with several other test scenes showed that the extent of memory usage reduction depends on scenes.  We hope the reported updates will allow branch users to render big scenes without an out-of-memory crash.

Weekly update November 15-28

The dev team was working on a bug fix during the last two weeks in close collaboration with Stéphane, the main developer of the original Freestyle program.  The problem was that the view map calculation did not take mesh face smoothness into account.

So far, mesh geometry data was treated as if all faces were flat (i.e., face smoothness was just ignored), which implied that the view map was composed of jagged feature edges as illustrated in the example below.  The left image is a rendering result of a torus mesh, where strokes are drawn along visible and hidden silhouette lines of the torus.  The right image shows the feature edges (highlighted) from a three-dimensional non-camera viewpoint.  These jagged polylines are what constitute the strokes that appear in the render.  As you see, the feature edges exactly lie on edges in the mesh geometry data.  This causes somehow jaggy strokes in the rendering result (especially in the part of the render where strokes make a quick turn).

 

When the face smoothness is taken into account in the view map calculation, the resulting feature edges totally differ from the outcome shown above.  As seen in the example below,  the three-dimensional feature edges representing the silhouette lines are much smoother.  The left image is a render of the same torus mesh, whereas the right image shows the smooth feature edges resulting from the view map calculated by accounting for the smoothness property of mesh faces.  Note that the smooth feature edges are no longer constrained by the original mesh topology.

 

As far as the dev team has experimented with several test scenes, taking account of mesh face smoothness results in a cleaner stroke topology.  The downside is that the visibility of smooth silhouette lines is mathematically not well-defined.  This means that rendering results may suffer from some artifact due to incorrect line visibility.  To our knowledge, the visibility of smooth silhouettes is still an open problem in computer graphics research.  Therefore, we decided to introduce an option for enabling/disabling mesh face smoothness in the view map calculation.  In the Freestyle branch revision 33350 and later, you have the new option “Face Smoothness” in both the Parameter Editor mode and Python Scripting mode.  When the option is enabled, the view map calculation accounts for face smoothness.  This option is intended to allow users to try the two ways of view map calculation with and without face smoothness and find which one would fit individual application needs.  For now the option is disabled by default in order to avoid the aforementioned visibility issue.

Weekly update October 25-November 14

The dev team was working on bug fixes and stability improvements during the last three weeks.

First, a fatal bug in the internal representation of curves (e.g., chained feature edges, and strokes) was fixed.  The bug was in the constructor of the CurvePoint class in the C++ layer, and in fact it was the source of several similar bugs.  For ease of future bug hunting, a fatal error in the CurvePoint constructor will result in a verbose error report in the console.  If you encounter a crash of this kind, please let us know.

Second, a bug in the SpatialNoiseShader was fixed.  The SpatialNoiseShader is used by standard style modules to add noise distortion to strokes.  The symptom of this bug was that some strokes resulting from the SpatialNoiseShader occasionally disappear from a render.  The problem was caused by a signed integer overflow in the PseudoNoise generator used by the shader.  When image resolution was very large (2000-by-2000 for instance), the buggy PseudoNoise generator could yield a series of huge noise values, which tossed the distorted strokes out from the image boundary.

Finally, a proper support for relative paths of style module files was implemented.  It is recalled that in Blender, file paths are either absolute or relative from the directory in which a .blend file resides.  When relative paths are used, it is a user’s responsibility to keep the relative paths valid.  To help you maintain valid relative paths, Blender offers you the “Remap Relative” option when you save a .blend file.  Suppose that you have created a new scene from scratch (based on the default scene after startup).  When you save the new scene for the first time, the default .blend file path will be $HOME/untitled.blend, where $HOME refers to a platform-specific home directory.  If you have relative paths in the new scene data, they are relative from $HOME.  Therefore, if you want to save the .blend file into a different directory, it is your care to update the relative paths accordingly (otherwise, the relative paths will no longer be valid).  The ”Remap Relative” option does these updates of relative paths automatically.  And from revision 33068 of the Freestyle branch, the “Remap Relative” option takes care of relative paths of style module files.  In addition, the following operations work properly with respect to style module files:

• File >> External Data >> Make All Paths Relative
• File >> External Data >> Make All Paths Absolute
• File >> External Data >> Report Missing Files

Weekly update October 18-24

In addition to the support for line caps, a set of GUI elements for dashed line was introduced in the Strokes panel of the Freestyle Line Style tab.

By enabling the Dashed Line toggle button, you can specify three pairs of dash and gap lengths in the unit of pixels.  Dash values define the lengths of dash strokes, while gap values specify intervals between two dashes.  If a zero gap is specified, then the corresponding dash is ignored even if it has a non-zero value.  Dashes are treated as separate strokes, meaning that you can apply line caps, as well as color, alpha and thickness modifiers.  The image below shows a few examples of dashed lines on the default cube.

Panel (a) shows simple dashed lines along hidden edges.  Panel (b) shows an example of dashed lines defined by two dash-gap pairs.  Panels (c) and (d) illustrate dashes with round caps.  Panels (e) and (f) are results of Along Stroke thickness modifiers applied to dashed lines.

Weekly update September 22-October 17

In the last blog post, we discussed how the complexity of internal stroke representations may increase when line joins, line caps and extreme thickness are taken into account.  After that, in the BlenderArtists Freestyle thread a separate discussion took place about Python scripting for implementing line caps.  This reminded the dev team that the implementation of different line caps is not that difficult and can be done independently of the other two issues of line joins and extreme thickness.  Hence, a new set of options for controlling the line caps has been added as shown in the screenshot below:

The Strokes panel of the Freestyle Line Style tab now allows you to use three types of line caps: Butt (flat), Round (half-circle), and Square (flat and extended).  See the following example images for each type of line caps.

  

Another small addition to the Strokes panel is the Same Object option.  By default, feature edges from different objects may be connected to form a stroke.  The Same Object option alters this default chaining rule, requesting that feature edges be connected only when they belong to the same object.

Weekly update September 1-21

Over the last three weeks, the dev team was working on two things.  One is integration of the new Parameter Editor mode into Blender’s animation system, and the other is a preliminary study on improvement of line drawing quality.

When we started adding the new graphical user interface (GUI) component, we had two implementation goals in mind: to make Freestyle accessible to artists without programming efforts, and to make Freestyle-generated line drawing animatable.  One of Blender 2.5 design concepts is to make everything animatable.  This rule applies to Freestyle as an integrated part of the 3D suite.  To this end, what we did is the following.  First, basic functionality for manipulating keyframes with respect to stylization parameters in Freestyle was implemented.  Keyframes can be specified in terms of base line color, alpha transparency, and line thickness, as well as all UI controls in color, alpha and thickness modifiers.  The following image is a capture of the context menu you get when you right click on UI controls in the Line Style tab.  Using this menu, keyframes can be inserted, replaced and deleted.

After we got the basic keyframe manipulation working, we moved on to extending animation-related editors in Blender, in order to permit manipulating animation data in line style data blocks.  Specifically, the Graph Editor, DopeSheet, and NLA Editor have been extended so that you can edit keyframe curves, drivers and actions with respect to line style parameters, as illustrated by the following screenshot.

The image below is a simple example of stylization parameter animation by means of the functionalities described above (plus postprocessing with GIMP).

Playing with animating strokes no longer requires any Python scripting.  It is a matter of imagination.  We hope you enjoy line drawing animation with Freestyle and Blender.

In the meantime, the dev team was investigating how to improve the quality of stroke rendering.  At the moment, stroke rendering in Freestyle does not have options for altering line joins (such as miter, round and bevel joins) and line caps (such as butt, round and square caps).  In addtion, strokes in Freestyle cannot be closed for now, meaning that every stroke has the beginning and the end.  These options are supposed to be available in every line drawing software, and indeed they are necessary in Freestyle as well.

A technical issue here is that strokes are currently represented by a specific mesh structure called triangle strip.  This mesh structure is easy to deal with, but its power of expression is very limited.  When complex strokes with line joins and line caps are considered, triangle strip is not necessarily applicable and a more general mesh structure is required.  Using a general mesh structure has its own difficulties.  For instance, generating a stroke mesh from a series of feature edges might be tricky when the stroke is thick and it makes a sharp U-turn.  The following image shows what Inkscape, a vectorial drawing tool, does when a path has different thickness values.

Red paths are identical, and grey polygons show the areas covered by the path with different thickness values.  This exemplifies what Freestyle should do when it draws a stroke along a series of feature edges.  When thickness values are relatively small as in (a) and (b), nothing special happens.  Increasing the thickness, however, makes the outline of strokes complicated, as shown in (c) and (d).

Branch users may have encountered a problem that unexpected long lines are occasionally drawn by Freestyle.  This is in fact a bug related to strokes with U-turns.  Fixing this issue is a priority task, and properly supporting line joins, line caps and closed strokes will be an integrated part of it.

More work on these stroke rendering issues as well as the new GUI development is anticipated in the next weeks.

Weekly update August 12-31

Since the last blog post, the dev team was working on several issues reported by branch users.  Two bugs causing a crash of the Freestyle renderer were successfully fixed, thanks to the provision of a .blend file by Cloud_GL for bug hunting.  One of the bugs was related to the use of movie file formats such as AVI, QuickTime, and FFMPEG codecs.  When one of these movie file formats is in use, animation rendering with the Freestyle renderer enabled was very unstable.  The other bug was in the Resampling shader for changing the spatial resolution of strokes.  Since the Resampling shader is widely used in standard style modules, the bug must have caused a lot of trouble on the users side.

Soon after the instability issues were addressed, several branch users kindly reported that the branch is more stable than ever and now is capable of rendering those .blend files that previously cause a crash.  We really appreciate the feedback, thanks!

In the mean time, Stéphane, one of the developers of the original Freestyle, sent us a link to a movie clip for introducing Freestyle.  The movie clip was screened during the Technical Papers Fast Forward session in SIGGRAPH 2010.  The entire movie clip was made using Blender and Freestyle.  Thank you Stéphane for sharing the nice video!

Weekly update July 12-August 11

Since early June, the dev team has been working on the new Parameter Editor mode that is intended to be an artist-friendly graphical user interface (GUI) for the Freestyle renderer.  A number of new UI elements have been introduced, and most of them are now considered functional.  Although the Parameter Editor mode is still a work in progress, a brief description of the new UI elements would be useful both from branch users’ perspective and for the purpose of GUI design review.

With the aim of organizing a number of stylization parameters in a clean and easy-to-understand form, two new notions of lineset and line style have been introduced. Lineset is a set of selected feature edges that are subsequently chained to create strokes.  Line style is a set of stylization parameters that define the appearance of a lineset.  Each render layer may have an arbitrary number of linesets, each of which is associated with a line style.  Line style is a new data block type in Blender, meaning that a line style is shared by different linesets (like the materials that can be shared by multiple objects).

A lineset accommodates a set of options for selecting feature edges on which you want to draw strokes.  First, you specify the types of selection criteria based on which feature edges are selected.  There are two types of selection criteria: edge visibility and edge types.

• By edge visibility, you can select all visible edges, all invisible (hidden) edges, or the edges within a range of quantitative invisibility (QI).  It is recalled that the QI of a feature edge is the number of occluding faces between the edge and the camera (i.e., the QI value is integer).
• Feature edges can also be selected by their types: silhouette, border, crease, ridge, valley, suggestive contour, material boundary, contour, and external contour.  Conditions on edge types can be combined by logical OR and logical AND.  In addition, there are two options named Inclusive and Exclusive.  By enabling the Inclusive option, the feature edges that satisfy the given AND/OR-combined edge type conditions are selected.  With the Exclusive option enabled, the feature edges that do not satisfy the given conditions are selected.

At the moment, there are no options for controlling how to chain selected feature edges to create strokes and how to split strokes into pieces.  New UI elements for chaining and stroke splitting will be added later.  For now, a specific chaining rule is used to convert selected feature edges to strokes.

A line style comprises a number of stylization parameters that define the appearance of strokes to be drawn on the feature edges selected by a lineset.  New UI elements for controlling line color, alpha transparency, and line thickness among many other stylization parameters are available in the Freestyle: Line Style tab.  A line style defines the base line color, base alpha transparency, and base line thickness.  In addition, these base values can be altered by modifiers.  Color, alpha and thickness are independently modified by a list of modifiers as shown below.  Modifiers are processed in the order within a list, namely in such a way that a base value is first modified by modifier #1, then modified by modifier #2, and so on.

Modifiers have two common properties: blend type and influence.  The blend type specifies the way how the modifier changes a base value, whereas the influence (on the interval 0 to 1) determines how much the modifier changes the base value.  An example of blend types is Mix to blend the base value and a new value based on the influence as follows.  Let B be the base value, X be the new value, and F be the influence.  Then, the base value is changed to B(1 – F) + XF.  If F = 1, then B is set equal to X.

As of this writing, there are three modifiers: Along Stroke, Distance from Camera, and Distance from Object.  These modifiers are all available as color modifiers, alpha modifiers, and thickness modifiers.

Along Stroke modifiers alter a base value (i.e., color, alpha, or thickness) by a new value that is defined along a stroke.  Suppose that a point moves along the stroke from the beginning to the end, and the location of the point is expressed by parameter T on the interval 0 to 1.  The Along Stroke modifiers yield a new value based on the T value.  The Along Stroke color modifier has a color ramp which is used to translate the T value to a new color, as illustrated in the figure below:

Both the Along Stroke alpha and thickness modifiers have a mapping that translates T to a new alpha/thickness value.  The mapping is either linear or curve.  The linear mapping is used for linear fade-out (see panel (a) in the figure below).  The linear mapping has the “invert” option for linear fade-in (see panel (b)).  The curve mapping allows you to specify a non-linear variation of new alpha/thickness values (see panel (c)).

The Along Stroke thickness modifier also has the Value Min and Value Max options that specify the range of the new thickness values.

Using all the Along Stroke color, alpha and thickness modifiers (with the curve mappings enabled for the latter two) gives the following render.

Distance from Camera modifiers change a base value by a new value that is determined based on the distance from the camera to a point lying on a stroke.  The distance is measured in the three-dimensional space and is expressed by parameter T on the interval 0 to 1.  The Distance from Camera color, alpha and thickness modifiers commonly have the Range Min and Range Max options.  When the distance from the camera to the point is equal to or less than the Range Min, the parameter T takes the value zero.  The T value approaches to one as the distance gets close to the Range Max.  The parameter T takes the value one if the distance is equal to or greater than the Range Max.

The Distance from Camera color modifier has a color ramp option for translating the T value to a new color, which is then blended with the base color (see panel (a) in the figure below).  The Distance from Camera alpha and thickness modifiers commonly have a mapping (either linear or curve) to translate T to a new alpha/thickness value (see panels (b) and (c)).  The Distance from Camera thickness modifier also has the Value Min and Value Max options to define the range of new thickness values.

Enabling all the three Distance from Camera modifiers above results in the following render:

Distance from Object modifiers are similar to the Distance from Camera modifiers, and the difference is that the distance is measured from a specified object (more specifically, from the center point of the object) instead of the camera.  Distance from Object modifiers have all options the Distance from Camera modifiers have, plus an entry to specify an object from which the distance to a point lying on a stroke is measured.

Enabling all the three Distance from Object modifiers above gives the following render:

That’s all for now.  More updates to the Parameter Editor mode are anticipated in the next weeks.  Feedback of any kind concerning the design and implementation of the Parameter Editor mode is much appreciated.

Weekly update June 28-July 11

Since the last blog update, the dev team was working on the design and implementation of the new GUI framework (referred to as the Parameter Editor mode), while important updates in the trunk were merged and some compilation issues were fixed.

The Parameter Editor mode has now a set of UI controls for better specifying feature edge selection criteria.  The Freestyle tab in the Render buttons has a pair of radio buttons named Inclusive and Exclusive, as well as another pair of radio buttons named Logical AND and Logical OR.  The Inclusive button allows you to select feature edges that satisfy the given selection criteria, while Exclusive is to select those that do not satisfy the given conditions.  The Logical AND and Logical OR buttons are for combining selection criteria by logical “and” and “or” operations (i.e., logical conjunction and disjunction), respectively.  These UI controls allow you to easily select desired feature edges that will be fed to subsequent stroke rendering.

The trunk has recently adopted a new installation path structure, and the Freestyle branch has already incorporated the path changes.  For branch users, this means that standard style modules are installed in a different path.  Old paths in your blend files may need to be fixed manually.

In the mean time, Stéphane Grabli, one of the coauthors of the original Freestyle, kindly provided us with a set of patches for fixing bugs that existed in both the original Freestyle and the Blender Freestyle branch.  One of the fixes is concerned with stroke construction, which is now capable of generating better feature edge topology thanks to more robust visibility tests.  The improvement is salient when you look at external contours.  The following image shows a comparison of external contours generated by the old revision 29733 and the new improved revision 30169.  The red and blue colors are intended to show where each stroke starts and ends, respectively.  The old version generated 55 strokes, showing lots of stroke fragmentation.  The new version yielded only 22 strokes (most of them were out of the camera view), demonstrating a much better stroke quality.  Thank you Stéphane for the wonderful contributions!

In the next weeks, we are going to keep up the work on the new GUI stuff.  Additions of new UI controls into the Freestyle Line Style tab are planned.