Rendering step complete

I completed last post’s work by optimizing the rendering process (directly copying the back buffer into the RenderResult object instead of manually copying the information). I am sure that is a good performance improvement.

I also understood why the first render had its color inverted: I first noticed that the result was only showing the object silhouette, while it should have also shown the object (as it is selected by default in the viewer’s configuration). I then remembered that Stéphane had mentioned that the rendering process is made of two steps, building the strokes and rendering, and I then looked at what that “rendering” function could be. I came to understand that the Controller::DrawStrokes() method is the one responsible for building up the strokes, while the final result is obtained by the AppGLWidget::draw() method. Combining both methods resolved the issue.

Here is the corrected render result that should be obtained:

I am happy to note that from this revision on (rev 15047), we have a rendering foundation to base the remaining work on. Even though it is now limited to a static scene, I consider that the hardest challenges of the first phase have been resolved. I am also delighted that the project is on schedule and going forward. Getting tangible results is a big motivation boost and I am going to continue to bring the project further. Right now, I am going to slow down on the features and make sure that it compiles well on Linux and Windows. If all goes well, I should meet the June 8th deadline for finishing the first phase.

Other results:

First render !

I managed to get a little bit further. I corrected the GLStrokeRenderer::preparePaper function to properly load the paper texture (it took me a while to understand that ImBuf has a 32 bit image structure). I also set up the viewer to fixed dimensions 640×640, finally allowing the strokes to be rendered.

When I render the scene, the object shows up very quickly (I recognize the teapot’s shape) but is automatically erased by a gray background. I tried to grab a screenshot or a screencast of it, but it’s unfortunately impossible to do so. Since I did not make any change to the way things are rendered, I am guessing that when the render frame window is instantiated, its OpenGL context is selected for rendering. I have to analyze a little bit closer what’s going on to keep the result on display.

Update:
By studying Yafray’s code in more detail, I understood why the result was not kept on display. I first needed to correctly initialize the freestyleRender function in the rendering pipeline (creating a RenderResult structure to hold the image result and passing the Render variable to Freestyle’s execution function). In Freestyle’s rendering function, after having drawn the strokes, I copy the rendered strokes from the back buffer into a float array and use that data to load up the RenderResult‘s image. The result is then drawn into the render window.

Even though this revision is a big improvement over the past steps, phase 1 is still not over with. I am currently experiencing a few notable issues:

• the paper texture is apparently not displayed (even though it is loaded) and the brush stroke color is inverted. Is that a Freestyle feature ? I do not know yet.
• the view map computation may crash for some models (simpleScene.3DS for example) and other style modules (not tested yet).
• Freestyle’s initialization is called each time the RENDER button is pressed, which could cause some memory leakage (the paper texture is reloaded each time without being deallocated at the end of the render).

After removing the fixed file references, I will need to change the UI to display a list of selectable style modules. Then, I will need to export the model data in 3DS format and the camera information to the viewer to finish up the first phase.

Here is a screenshot of what the first render looks like (the teapot is also upside-down in the original 3DS file):

Style module execution works !

I think that I resolved most (all ?) of the Python/SWIG related issues today. First, I updated yesterday’s PythonInterpreter and Controller code to correctly initialize the interpreter and the canvas. Second, I managed to link the SWIG wrapper properly and the logs show that the different Python modules are correctly called. Unfortunately, because of my limited understanding of scons, I haven’t found a way to automatically create that SWIG library. In the mean time, it can be generated from source/blender/freestyle/intern/swig using the two commands found in the SWIG section of source/blender/freestyle/SConscript. As a note, the following configuration does not use SWIG to generate the wrapper. Once I am sure that it compiles on different platforms, I will remove the swig compilation step (in extern/freestyle/swig).

I hope that all Python bugs are already a thing of the past. From my understanding of Freestyle’s rendering pipeline, I am already halfway through the final rendering step (brush strokes have been generated by the style module execution). Tomorrow, I’ll take on the final part of that step, using the render window OpenGL context to render the strokes and display the result. I have no idea how easy it is going to be. I hope to have it finished in two days.

Once this is working, I’ll have to remove all the static information currently used (fixed model, fixed style module…) and replace it with the scene information. I hope that process will go smoothly.

PythonInterpreter bug fixed using Blender's native Python environment

I was stuck for a long time today on PythonInterpreter‘s bug. Using gdb, I was able to trace down why it was crashing. Whenever a command was sent to Python using the C interface function PyRun_SimpleString, it would in turn call PyImport_AddModule( "__main__"). For some reason, this command would crash with the "No such file or directory" error. Of course, this error should not happen. __main__ (among others like __built__) is supposed to be initialized with the Py_Initialize() method. It was a weird error. I had made sure that Python was well initialized, but it hadn’t resolve anything.

Understanding that the nature of the problem was probably beyond the project’s interest (and my means…), I decided to use Blender’s Python functions directly (which is planned to be done at phase 2 of the project). I am surprised to see how quick the integration was: I used bf_blenkernel‘s add_text method to load the file into a Text structure, bf_python‘s BPY_txt_do_python_Text method to execute the (fixed) script file, BPY_Err_getLinenumber to display the error line number. The Python script is executed and stops because it doesn’t find the Freestyle module.

Next, I am going to compile the SWIG module wrapper and integrate it so that the Python interpreter can use it to call the Freestyle API functions.

Update:
The code was updated to properly handle PythonInterpreter‘s initialization, following Freestyle original code.

View map calculation finally works, Python crashes

Since last post, I have continued integrating libQGLViewer’s code and it’s been a painful process: I have wasted a lot of time fixing recursive and Qt dependencies. Until today, I was still working on migrating that code. I decided to backtrack a little to concentrate on only the strict necessary, being a little more selective. I don’t know whether I was lucky or not but that seemed to work:

• I kept the Camera, Constraint, Frame, ManipulatedCameraFrame, ManipulatedFrame, Quaternion, Vec classes
• I introduced a Point class, based on QPoint
• I removed all remaining Qt-specific information (slots, QOBJECT….), interaction-related code (mouse and keyboard code) as well as the keyFrameInterpolator methods

View map building executes smoothly now, looking at the logs (and it doesn’t crash…). The issue seems resolved. Of course, all of the above classes will have to be replaced in the future; I am just happy not to have to do it now.

I tried to move on to next steps (drawing the strokes). The Python interpreter seems to crash as soon as it is sent a command (Python initialization seems fine). My crash dump lists:

0 org.python.python 0x0168eb27 PyImport_AddModule + 22 (import.c:320)
1 org.blenderfoundation.blender 0x00ae4241 PythonInterpreter::initPath() + 65
2 org.blenderfoundation.blender 0x00ae463d PythonInterpreter::interpretFile(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 29
3 org.blenderfoundation.blender 0x00aa4fa3 Canvas::Draw() + 755

I have spent the rest of the night trying to figure out what could cause the crash but haven’t found any real information. I tried including __main__ before importing the module sys, like I saw recommended in certain forums but it did not change anything. I am not sure whether the crash is due to Python, to Blender’s environment or to other reasons. I am confronted with the earlier choice of trying to integrate Blender’s Python environment. Should I do that ? Do I really have a choice ? I don’t know what’s best for me.

I am also interested in integrating the swig wrapper but I am not sure about how to best go about it. Right now, I have compiled it as a shared library (as it is done in Freestyle) and made it available via the Python path. I wonder if linking conflicts are going to appear, since Freestyle’s code in Blender is linked as a static library (while Freestyle as an application is linked with a series of dynamic libraries). Should I still compile the wrapper as an independent dynamic library ?

Lib3ds works, libQGLViewer replacement from its source code

After finally having Freestyle compiled with the previous code change, I got a little farther in the first phase. The whole process of creating a render is there, it is still not functioning because libQGLViewer’s functionality has not been totally replaced.

The controller execution goes as follows (Stéphane, tell me if I am missing something):

1. instantiates the config path, the controller and the view
2. sets the controller’s view
3. loads a 3ds file (right now a fixed file)
4. inserts a style module (right now, also fixed)
5. computes the view map
6. draws the image

I am currently on step 5. In Freestyle’s code, this step depended on the libQGLViewer widget to obtain the necessary camera information for raycasting calculations. I had originally replaced the Camera class with an empty stub, but that led to view map calculation crashes. I have no choice but to also merge the Camera and Frame code from the libQGLViewer source (that’s thankfully available). The problem that I am currently facing is that merging the code has a ‘snowball effect’ on the required dependencies: Camera requires manipulatedCameraFrame, which requires manipulatedFrame, which requires Frame…. You get the picture, I am going to have to integrate most of libQGLViewer’s code !

The only good news of the night is that model and style module loading runs smoothly without a hitch (I am surprised how lib3ds is working out of the box). I know I am getting closer to a render, even though I am not totally there yet. Besides finishing the libQGLViewer migration, I also need to figure out where to find the required OpenGL context needed for the actual render

Patience, patience, patience….

Stuck at the linking stage

I am currently at a halt because of linking issues: gcc cannot resolve the dependencies. Apparently, because of the current linking order, Imbuf functions within my module are not recognized and the final linking stage for the blender program fails. I spent the whole night yesterday trying every possible combination, but to no success. I have asked jesterKing to help me on this and because of compilation issues under Windows, we were not able to work on my current problem.

mfoxdogg also did a compilation under linux today and a few unexpected errors were found. Particularly, one error is pretty worrying: his configuration did not recognize the OpenGL glBlendEquation function. That’s of course totally out of my reach, I don’t expect to know how to correct and support these low-level issues. What should I do ? I asked him to comment the related parts but that’s of course a temporary solution.

I am going to wait for jesterKing’s feedback first. If you could try to compile the current revision and tell me what bugs/warnings/errors you encounter, that would be greatly appreciated.

Update:
Thanks to Jean-Luc, the problem was resolved tonight ! Jean-Luc analyzed the linking order and understood that regardless of the position of the imbuf library in the linking process, nothing would change. He then looked inside freestyle’s library and realized that Imbuf’s functions mangling names were of C++ type (while Imbuf is coded in C). By putting extern "C" {...} around Imbuf’s headers, gcc was able to find the functions in the libbf_imbuf.a library and linking is now working again. Now, I have to debug the Freestyle crash within Blender…

Substituting libQGLViewer

I am currently trying to generate a first render and the process is of course more complicated than the previous steps. While Freestyle’s non-UI code is pretty much self-sufficient, the UI parts are unfortunately not; there is high coupling between the functions demanding a render and the methods actually producing the images. This is of course the challenge of the first phase and I am not surprised that it is not as easy as I thought.

I started by modifying Blender’s render interface to call Freestyle’s Controller methods. I originally wanted to make Freestyle independent of libQGLViewer, constructing an image using the Canvas class. I have come to understand that it is not possible: libQGLViewer is at the core of the rendering process, hence I must not remove it but substitute it. I have to replace much of AppGLWidget‘s functionality and I need to reimplement the basic methods and classes that it is based on (QGLViewer). On top of that, I need to continue removing QString and QImage dependencies.

I have not finished correcting all the compilation errors. I am going to continue all of this week-end and I hope to generate an image by tomorrow. Of course, the big unknown is how much time will be required to replace libQGLViewer’s methods. As I am replacing them, more and more dependencies come up. As soon as I have an image generated, I will commit my progress.

Freestyle core compiled

I have successfully migrated all the Freestyle code directories (except the app/ folder), under my configuration. I also took care of resolving compiler warnings so Freestyle’s library should compile warning-free (under gcc4 at least). I replaced QImage by ImBuf and QString by the standard library string. For example, I changed QImage’s image construction QImage image(filename) to ImBuf *image = IMB_loadiffname(filename, flags). Unfortunately, I haven’t been able to test if these code changes really work. Now that Freestyle’s core is compiled, I can move on to replacing the GUI controller-based execution.

As a note, jmsoler tried to compile the latest revision on a Win32 config and unfortunately, because the SConscript depends on the configure script to determine what files to compile, it doesn’t work. I just removed that dependency for lib3ds, but I’ll need to do the same for the swig compilation, which is also based on configure.

Update:
SWIG’s website gives (and recommends using) precompiled Win32 binaries. I added that win32 binary to my repository and changed swig’s SConscript to make a simple copy to the build directory in that case.

Towards Freestyle's compilation… let the pain begin !

I just finished resolving the first set of dependency problems, compiling lib3ds and swig as independent external libraries. That is of course no real achievement… Nevertheless, this last two days were important for me, as I learned how to use Blender’s SCons building system in practice. I have a rough idea of how configuration, compilation and linking works.

Jean-Luc, I am going to follow your advice and set up an imbuf-based canvas remplacement. I am first going to study how it is used within Blender, and I’ll go from there.

You mentioned that I can use the compiled swig library as a binary. Can you show me how to do that ? I’ll need to run the equivalent of the following command: swig -c++ -python -o ModuleWrapper.cpp $<

Update:
I actually reverted back to my initial solution: I am compiling swig as a program, rather than a library. I also had to modify the BlenderProg function in Blender.py to circumvent the .app creation process. I am aware this is a hack, but it’s temporary. When Freestyle becomes SWIG-free in phase 2, I’ll remove it.