Style module selection in Freestyle panel

Finally a big usability improvement ! You can select the style module for your own tests, without having to recompile Freestyle… Right now, only the style module pathname textbox and the file selection button are available. I’ll try to add more of the original Freestyle preferences if it is useful for the project:

As you should see it in your own test logs, not all style modules execute properly. This is due to either the shade method not yet functioning correctly or the API not taking into acccount all of the possible constructor definitions. This should be fixed soon.

Cross-language polymorphism support

I wanted to jump right on the lib3ds work before testing the new system further, but bad intuition kept me from doing that. I got intrigued by the errors I had gotten when testing and I was not content with the current work-around. Testing the system, I understood that the system was (read: is) unstable and unfinished. I realized it was missing an important feature, that Stéphane had mentioned very early in the project: cross-language polymorphism.

Basically, cross-language polymorphism allows the system to automagically support implementations in both language. If a C++ implementation is not present, the system tries to use the Python version if it is available. SWIG supports this feature through directors. To replace this feature, I had to study in more detail how directors are implemented in SWIG and how I could “recreate” them easily within our system.

The same way that I created a proxy mechanism to support Python to C++ redirection, I decided to support directors by having C++ objects store a reference to the Python object owning them. This gives  them the capability to “redirect” some of the execution back unto Python methods. If a C++ object uses the interface class method at runtime (instead of its own implementation), it will use the related Python object’s method if it is available, else will print an error message.

So far, the contour style module works correctly but style modules with shading function in Python do not work yet. To ease testing, I will (finally) have the style module selectable from the UI.

Freestyle runs natively in Blender's Python environment

Great news ! The SWIG replacement is effective as of today and seems to run fine. I am relieved that all of the hard work I have put in the last two weeks has paid off. I do not know how stable the new system is yet, but I can say that at least it works. I will continue to test it this week and improve it. At least, a good foundation is already in place.

I will start working on getting lib3ds removed today. It’s unclear how difficult the job will be. I can only hope that it is faster than removing SWIG (and it should, I don’t have the same scale of code to produce). The quicker I get that done, the sooner I can make my GSoC project interesting again: integrating Freestyle with the internal renderer. That should then allow me to support render layers. I also wish to add a Freestyle tab in the UI this week, to finally be able to select style modules in the interface.

Even though there is still a lot more work to do, today is still a big achievement for me. For the first time, my work should be compiled on Windows stations without too many problems. I will try to contact platform builders to make sure my work is stable accross different systems. More on that in the next days too.

Almost done with SWIG…

Sorry again for the recent lack of communication on the project. All of my progress of the past two weeks was logged in my Subversion branch commits. In case you want to have access to the latest updates, you can syndicate to the RSS feed available from CIA.vc.

I have been concentrating all of my efforts on getting SWIG removed as quickly as possible. Unfortunately, I have had to reconsider most of my original ideas. Instead of porting only 30 base classes and recoding everything in Python, I decided to port all of the original classes (about 140…) and keep most of the original API intact. I reconsidered my original plan to replace all iterators by list-based traversals. I would have had to either combine both functionalities into one data structure, or rewrite all the areas using iterators.  Not feasible “in a few days”…

Anyways, the good news is that it is almost over with. I should be able to test my work with original style modules tonight or tomorrow and if all goes well, I’ll move on to removing lib3ds next. If you are getting frustrated that you can’t  use Freestyle yet, you’ll have to continue to be patient :) Getting SWIG and lib3ds out of Freestyle are crucial to consider the GSoC project successful. I also need to remove all OpenGL bindings to consider the project complete. As I discussed it with Jean-Luc, I am not sure time will permit integrating Freestyle in the composer by the end of August. It really depends on how smooth the removal of lib3ds and the integration in the renderer goes. I will continue to keep you all updated on the blog.

SVertex, CurvePoint migrated

I was unavailable yesterday so I could not advance on my work or give feedback. Sorry about that.

Today, I almost finished implementing the SVertex and CurvePoint today. It is not totally finished due to a problem (bug ?) with the CurvePoint(CurvePoint *iA, CurvePoint *iB, float t2d) constructor. I’ll see with Stéphane about the problem.

I corrected the Python classes to allow rich comparison, providing the ability to override the default comparison operators. I also modified the C to Python conversion utilities, allowing more general cases and better exception testing. I provided some of the classes with the __copy__ method (known as ‘dupplicate’ in Freestyle).

I have not ported the userdata variable in some classes yet, not really knowing if it should be given in the API. Also, whenever the API should provide a result as a set, I’ll provide it as a list. The reason is that the STL and Python PySet implementations differ greatly: the STL version is based on a binary search tree (you just need to define a operator< on the underlying type), while the Python one is hash-based (you need to associate a unique key to each element). It doesn’t really matter for our API: C++ will no matter what keep it as a set and assure the integrity. The list type will just be used for access.

Proof of concept for SWIG replacement

I continued working on SWIG removal today. Even though I have no new class available, I improved the code quite a bit.

First, the Freestyle API classes are now (properly) available as module objects, instead of submodules. I had to incorporate object lifecycle-related methods. I first started adding a custom __new__ method. I understood later that the Python object type slot tp_new should be set to the standard PyType_GenericNew function, while it is __init__ that had to be customized (as is commonly the case for regular Python classes). I learned today that, by default, a PyTypeObject‘s tp_new slot is not available. It is crucial to define it for all base classes.

Second, I verified that my approach for replacing SWIG should work:

• I wrap each C++ base class with a PyObject of the same name (stored in the corresponding object type’s tp_name slot): the PyObject just contains a pointer to an instance of that C++ class.
• I redirect calls from Python to C++ by merely calling the corresponding methods on the C++ instance stored in the Python object.
• I bring the data back to Python by converting the C++ result to the appropriate PyObject (see source/blender/freestyle/python/Convert.{h,cpp}).

As a proof-of-concept, I started implementing CurvePoint (Interface0D subclass). I added CurvePoint‘s initialization method, which sets the C++ Interface0D pointer if0D to the address of an instance of the C++ CurvePoint class. When I test an instance of each class with the getExactTypeName method, I obtain the proper result (their name !).

So far, BinaryPredicate0D, BinaryPredicate1D, Id, Interface0D and Interface1D have been implemented (they are available in Blender’s Python environment under Blender.Freestyle). I’ll update the blog again tomorrow to give you the list of available classes.

Midterm report

I uploaded my GSoC midterm report on the Blender wiki. It is nothing really new, just a summary of the areas that are complete, those that still need to be worked on and what challenges the project is facing.

On the SWIG replacement front, I will explain how it’s working when I am done with it. I do not want to dilute my efforts into documentation while there are still essential features that need to be implemented. I’ll also have the last two weeks of August to focus on documentation.

Quick feedback on SWIG removal advancement

I am very happy with my progress today. I have a greater understanding of how SWIG and the Python interpreter in Blender works. I feel like I have overcome the knowledge gap necessary to not only implement Freestyle’s Python API, but do it right. For short, I plan on implementing the bare essentials via the Python/C API and transfer the majority of the code in Python (currently done automatically by SWIG). I’ll remove all (currently strongly-typed) iterators for a list-based iteration and I’ll do away with all class casting code. I would want to elaborate on these details but frankly, I am exhausted… I have to get some sleep. I’ll try to describe my approach over the week-end, to make sure that I am getting this right. The resources that I found particularily helpful were:

• Interfacing C/C++ and Python with SWIG
• Extending and Embedding (especially Subclassing other types)

On a totally irrelevant note, this blog received its first spam comment today. That’s something to celebrate.

Approaching SWIG replacement

I started filling up some classes today and I am finding out that the process is not as obvious as I thought. The major problem is that I am building up the Python support incrementally, with no real “vision”. Having read the way Blender’s Python API was coded in the past few days, I am trying to follow the same standards but I am not sure it is the way to go (especially on error handling). The lack of a clear API means I do not have the data structures to work with and I am challenged by what parts to keep.

The process is no doubt tedious and repetitive. I feel like I am keeping a badly toned-down version of SWIG, contracting the wrapper code it produced. Things get even more complicated with class inheritance: I would have to copy the code from the ancestors, with some slight modifications. I need to figure out with Jean-Luc how to best go about this migration. I hope we will have time to decide on an API and data structures tomorrow.

Finally, I gave the render buffer another shot. Besides continuing to play around with flags, I rewrote the code from scratch. I noticed that, under Blender, I would get crashes when I created buffers/textures attached to “different” attachment points (COLOR_ATTACHMENT2_EXT for example). Using glCheckFramebufferStatusEXT, I realized that the buffer drawn to and read from must be set to the one specified by that attachment point. Nevertheless, rendering to that renderbuffer corrupts the back buffer and the result image. If anyone is able to have frame buffer objects working directly within Blender’s rendering pipeline (a great example would be rendering a simple quad to a texture and copying the resulting texture back into a render layer’s float rect), I would be happy to try it on my machine.

SWIG replacement setup

I started implementing the SWIG replacement today. So far, it’s pretty empty… Based on the information available on Freestyle’s original Sourceforge site, I first recreated the class hierarchy. I originally wanted to manually go over each class’ interface and recreate each one of them. I realized that I could potentially miss some functions here and here. Since SWIG has worked without a hitch, I decided to reuse the wrapper files it generated. Using some ugly regular expressions, I was able to extract almost all of the wrapper functions’ definitions which will serve as a canvas for the implementation. I can’t really estimate how long it is going to take me to finish the task. I hope to have something to test in two days.

If you have tips that could help me on the renderbuffer problem, please do contact me. Any help is appreciated.