Freestyle integration into Blender

September 23, 2010

Weekly update September 1-21

Filed under: Update — The dev team @ 1:27 AM

Over the last three weeks, the dev team was working on two things.  One is integration of the new Parameter Editor mode into Blender’s animation system, and the other is a preliminary study on improvement of line drawing quality.

When we started adding the new graphical user interface (GUI) component, we had two implementation goals in mind: to make Freestyle accessible to artists without programming efforts, and to make Freestyle-generated line drawing animatable.  One of Blender 2.5 design concepts is to make everything animatable.  This rule applies to Freestyle as an integrated part of the 3D suite.  To this end, what we did is the following.  First, basic functionality for manipulating keyframes with respect to stylization parameters in Freestyle was implemented.  Keyframes can be specified in terms of base line color, alpha transparency, and line thickness, as well as all UI controls in color, alpha and thickness modifiers.  The following image is a capture of the context menu you get when you right click on UI controls in the Line Style tab.  Using this menu, keyframes can be inserted, replaced and deleted.

After we got the basic keyframe manipulation working, we moved on to extending animation-related editors in Blender, in order to permit manipulating animation data in line style data blocks.  Specifically, the Graph Editor, DopeSheet, and NLA Editor have been extended so that you can edit keyframe curves, drivers and actions with respect to line style parameters, as illustrated by the following screenshot.

The image below is a simple example of stylization parameter animation by means of the functionalities described above (plus postprocessing with GIMP).

Playing with animating strokes no longer requires any Python scripting.  It is a matter of imagination.  We hope you enjoy line drawing animation with Freestyle and Blender.

In the meantime, the dev team was investigating how to improve the quality of stroke rendering.  At the moment, stroke rendering in Freestyle does not have options for altering line joins (such as miter, round and bevel joins) and line caps (such as butt, round and square caps).  In addtion, strokes in Freestyle cannot be closed for now, meaning that every stroke has the beginning and the end.  These options are supposed to be available in every line drawing software, and indeed they are necessary in Freestyle as well.

A technical issue here is that strokes are currently represented by a specific mesh structure called triangle strip.  This mesh structure is easy to deal with, but its power of expression is very limited.  When complex strokes with line joins and line caps are considered, triangle strip is not necessarily applicable and a more general mesh structure is required.  Using a general mesh structure has its own difficulties.  For instance, generating a stroke mesh from a series of feature edges might be tricky when the stroke is thick and it makes a sharp U-turn.  The following image shows what Inkscape, a vectorial drawing tool, does when a path has different thickness values.

Red paths are identical, and grey polygons show the areas covered by the path with different thickness values.  This exemplifies what Freestyle should do when it draws a stroke along a series of feature edges.  When thickness values are relatively small as in (a) and (b), nothing special happens.  Increasing the thickness, however, makes the outline of strokes complicated, as shown in (c) and (d).

Branch users may have encountered a problem that unexpected long lines are occasionally drawn by Freestyle.  This is in fact a bug related to strokes with U-turns.  Fixing this issue is a priority task, and properly supporting line joins, line caps and closed strokes will be an integrated part of it.

More work on these stroke rendering issues as well as the new GUI development is anticipated in the next weeks.

12 Comments »

  1. Wow, great job! I can’t wait to animate some of these parameters. The artistic freedom is incredible, well you all know that…. Thanks so Much!

    Comment by nrk — September 23, 2010 @ 3:31 AM

  2. Hello, congratulations on your great work

    One possible solution for rendering strokes is use a specific library (vector representaton) for this (cairo-Graphics or antigrain) this avoids transform lines on meshes (saving time and memory) and also the possibility of working with vector lines and advantages to export strokes (SVG,Eps)

    regards

    Comment by Ciriaco — September 23, 2010 @ 7:03 AM

  3. 2D open-graphics libs:
    http://www.antigrain.com/
    http://cairographics.org/

    =:-)

    Comment by Ciriaco — September 23, 2010 @ 7:17 AM

  4. (c) seams good to me.
    Indeed I would expect (d) to be better.

    Congrats for the animation work.
    Looking forward to a near integration with trunk :)

    Comment by Steren — September 23, 2010 @ 1:30 PM

  5. Thank you all for the comments!

    Ciriaco,
    We discusseed the possibility of using external 2D graphics libraries when we were trying to remove the OpenGL depenendency the original Freestyle program had. After a careful review of options, we decided to use the functionality of the internal renderer to achieve stroke rendering in Freestyle. A downside of relying on external 2D graphics libraries was to introduce another dependency instead of the OpenGL depenency. Blender has a lot of external packages on which it depends, and the build process is already complicated enough. We didn’t want to make it further complicated. See the following blog post for more information on this topic:
    https://freestyleintegration.wordpress.com/2008/11/28/close-to-removing-opengl-based-rendering/

    As the Freestyle integration project proceeded, we have been verifying this decision was right. For instance, we had to make full sample anti-aliasing (FSA) functional with Freestyle (see blog posts in March 2010). The internal renderer has a built-in support for FSA, and our task was to extend the existing code to properly merge the stroke rendering results into the final “combined” images. The task itself is easy to describe, while the implementation took a while due to the source-level complexity of the internal renderer. If we were using an external 2D graphics library for stroke rendering, a proper FSA support in Freestyle might have been a nightmare.

    On the other hand, I have been learning a lot from the existing 2D graphics software with regard to thick stroke generation. I am sure the 2D graphics packages you mentioned provide us with useful information and a working code base on the matter. Thank you for the suggestion anyway.

    Steren,
    I realized that I did not mention the fact that the last image was created using Inkscape. The image was intended to show what Inkscape does when a path has different thickness values, especially when the thickness is extremely large. All the four examples of the strokes are thus considered correct, exemplifying what Freestyle is supposed to do. I made this point clearer by just modifying the text of the blog post. Thank you for the feedback.

    Comment by The dev team — September 23, 2010 @ 10:03 PM

  6. very cool good work guys.

    I liked how you explained in lame terms some of the technical difficulties you guys are facing.

    I know it’s a feature that is way in the future, but i was wondering if at one time you could talk briefly about intersection lines and what limits and things you face in that area. It wouldn’t be interesting to me and perhaps people can give good sugestions. I’m not a coder, and my brain hurts thinking how to relate code to actual complicated product such as 3d apps and renderers and all that. I’d just like know what you guys have found out about it thus far and if you think it will be even possible since you said it’s a limitation of the original renderer.

    anyways awesome stuff, thanks for updating!

    Comment by Ike — September 24, 2010 @ 12:58 AM

  7. it WOULD be interesting to me***** not sure why i typed “wouldn’t” (more effort to tpye)

    Comment by Ike — September 24, 2010 @ 1:37 AM

  8. I am surprised by the look of (c) and (d), is this really correct? I am not completely sure, how I would want it to look, though, but a bit more convex, I think…

    Comment by yoff — September 24, 2010 @ 3:27 AM

  9. Thanks for clarifying the answer. I hope that “freestyle” will merge with the trunk soon.

    Congratulations for your great work

    Comment by Ciriaco — September 24, 2010 @ 7:25 AM

  10. Great work on the animation integration \o/

    For some reason (d) looks not right to me. (a), (b) and (c) seem to make more sense to me. I’m guessing there is some union/intersection issues.

    Also, the right-most slanted edge is *not* perpendicular to the short red line! You can see this clearly in (b), (c) and (d). It makes sense that this is because of the union of horizontal block stroke(due to the horizontal red line) with the stroke of the little red line. Even after assuming this (d) makes no sense, i.e. the small gap on the upper left should not be there!

    Anyway, this is great progress. I will force myself to wait for trunk integration :D

    Comment by dodobrain — September 29, 2010 @ 6:10 AM

  11. I was kind of surprised you weren’t using a third-party library for the 2D rendering as well, but if you’re making improvements to the 2D capabilities of Blender’s internal renderer then that’s great as well. I’ve always wanted to be able to use Blender for 2D vector animation as well as 3D, so this might be a step in that direction.

    Also, since you appear to be having success in showing real time previews of the freestyle edges, do you think it would ever be feasible to incorporate freestyle as real-time effect in the Blender Game Engine?

    Comment by Rudiger — October 15, 2010 @ 6:11 AM

  12. Ike,
    Excuse me for the lack of response for a long time. Indeed, the addition of feature edge detection at mesh intersection has been in the to-do list of the Freestyle integration project since last December. Finding mesh intersections is a well studied topic and there are lots of algorithms and their implementations to deal with it. So, from a technical perspective, I personally don’t expect much trouble in getting this new feature implemented in Freestyle. The main problem is time. I really hope I will be able to find enough time to work on it.

    Rudiger,
    Making Freestyle a real-time effect in the BGE sounds an interesting idea. I would say that is feasible, but it may require a substantial effort on performance optimization of the Freestyle renderer. At the moment, the most time-consuming part of the renderer is the view map creation in which feature edges are imported from mesh data in Blender and their visibility from the viewpoint of the active camera as well as edge types (such as crease, border, and material boundary) are computed. Importing mesh data from Blender to Freestyle every frame imposes a significant overhead. Maybe the two worlds need to be integrated more tightly in order to enable real-time performance. By the way, you might want to take a look at uimac’s work on real-time edge line effects. See FEDB’s example scene http://fedb.blogzine.jp/blenderclip/2009/02/viewedge_e6d7.html.

    Comment by The dev team — October 18, 2010 @ 2:12 AM


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: