Latest Entries »

In Ton’s recent blog post, he discussed a roadmap for Blender 2.7, 2.8 and beyond, which included a more tighter integration between Blender and the BGE. While this initially caused quite the stir in the BGE community with some thinking this meant dropping the BGE entirely, I see it more as a desire to get the two to share more code. Blender has smoke, fluids and particles, why shouldn’t we use those in the BGE? Too slow? Then lets speed them up and make Blender users happier in the process. The way I see it, the BGE can benefit from new features in Blender and Blender can benefit from performance improvements from the BGE. But, how do we get there? That’s what I aim to discuss in this article.

Sharing Blender Data

The first major problem that needs to be tackled is how the BGE handles Blender data. Currently, one of the BGE’s major design decisions is to never modify Blender data. While the BGE does modify Blender data in a few places (most notably lights), we’ve mostly stuck to this design principle, which has helped prevent numerous bugs and potentially corrupting users’ data. However, in doing so, we’ve had to recreate most of Blender’s data structures and convert all Blender data to BGE data. This also limits how we can interact with existing Blender tools. Blender has a lot of powerful mesh editing tools, but we can’t use those in the BGE because they require a Blender Mesh object while the BGE has a RAS_MeshObject, and using the original Blender Mesh would cause that data to change.

If we want a tighter integration between Blender and the BGE, we need to allow the BGE to have more direct control over Blender data. This means we need to find a way to allow the BGE to modify and use Blender data without changing the original data. The most obvious method is to give the BGE a copy of all of the data and then just trash the copy when the BGE is done. However, I think there is a bit more elegant solution to the problem. If you look at the existing code base, you can see that the Blenderplayer actually doesn’t have to worry about modifying Blender data as long as it never saves the Blendfile it reads. Only the embedded player has issues because it is using the Blender data already loaded in Blender. So, why not have the embedded player read from disk like the Blenderplayer? When the embedded player starts, the current Blendfile could be saved to disk and then loaded by the BGE. There are some details that have to be worked out here though, such as where do we save the file? A temporary location (e.g., /tmp)? That will cause path issues in larger games. Instead, I see two feasible locations: the original file or the original file appended with a “~”. The first would behave like a compiler would, you save before running your program, and is the approach I prefer. However, this changes the current behavior, which might upset some users.

A more long term solution to the problem of modifying Blender data is to drop the embedded player. As I mentioned before, the Blenderplayer doesn’t run into issues using Blender data since it doesn’t share a memory space with Blender. And, since the Blenderplayer supports being embedded into other applications, we can still have games running in what appears to be the viewport. In other words, we would not lose features! Some benefits to this approach:

  • Get rid of a lot of code (the whole source/gameengine/BlenderRoutines folder)
  • A lot less duplicate code
  • Smaller Blender runtime size (all BGE code would only be in the Blenderplayer, and not Blender)
  • Playing the game in the viewport and the Blenderplayer would be guaranteed to be the same (right now small differences exist)
  • The ability to modify Blender data without breaking Blender
  • A BGE crash won’t affect Blender since they will be in separate processes (like Chrome tabs)

However, there are some downsides, which include:

  • It will be more difficult to affect the BGE from Blender. At the moment this isn’t a problem, but if we want some goodies like Unity offers with adjusting the game using the editor while the game is running, we’d need to develop some inter-process communication protocol to get Blender and the BGE communicating.
  • We currently don’t allow embedding on OS X. I’m not sure if this is a limitation of OS X itself, or a lack of development effort on our part.

Using Blender Data

So, we’ve got some ways to minimize the issues of the BGE using Blender data, but what do we do with it? First off, I’d start to clean up the BGE code to use DNA data as storage and then shift the focus of the various BGE classes to act as wrappers around that storage. Where possible, the member functions of those classes could delegate to the various Blender kernel (BKE) functions. Once that is done, we can look into what Blender goodies we can start adding to the BGE using these new classes.

Viewport Drawing

While the BGE and Blender already share a fair amount of viewport drawing code (especially in GLSL Mode), this area could be much improved. The first task here is to get all of the OpenGL (and any calls to bf_gpu) into the Rasterizer, and only the Rasterizer. This requires moving material and lighting data out of Ketsji and into the Rasterizer. Once this is done, we can worry about how the BGE handles it’s drawing. The Rasterizer should have two modes (possibly implemented as two Rasterizers): fixed function pipeline and programmable pipeline. To do this, I would propose dropping Singletexture and making Multitexture code the basis for the fixed function Rasterizer, while GLSL mode would be the basis for the programmable Rasterizer. The programmable Rasterizer could have an OpenGL minimum of 2.1 as Ton suggested for his proposed roadmap, but I’d keep the fixed function Rasterizer as compatible with older hardware as possible.

After we have the Rasterizer cleaned up, we can start offloading as many tasks as possible from the Rasterizer to the bf_gpu module, which the viewport code also uses. The more we can put into this module, the more Blender and the BGE can share viewport drawing. Ideally, the Rasterizer would not have any OpenGL code and would rely entirely on bf_gpu, maximizing code reuse and sharing.

Conclusion

Using the ideas outline in this article, we’d have two main points of interaction between Blender and the BGE: BKE and bf_gpu. We could certainly look into more ways to increase integration between Blender and the BGE, but what I have discussed here will give us more than enough work for the foreseeable future. Also, please note that this is only a proposal and a listing of ideas, and by no means a definitive plan. Discussion and feedback is much encouraged and appreciated.

Bgui 0.08 Released

I’m looking to make some breaking changes to Bgui’s API, so I figured it was about time for a 0.08 release (especially since 0.07 was released over a year ago). Some cool changes that people might like include font outlines, a better animation system, and more control over font theming for the TextBlock and TexInput widgets. For more details on the changes, take a look at the changelog.

You can grab the new version from here.

Requirements:

  • Blender 2.6+ (tested against 2.66a)

Haven’t tried out Bgui yet? There is a Getting Started guide in the wiki.

Cheers,
Moguri

Well, this is it, my last report for GSoC 2012:

 

What did you do this week?

I cleaned up the Pre Z code a bit more. For now I’m only doing the Pre Z for GLSL Materials since the goal is to minimize time spent on fragment shaders. However, this means that Multitexture and Singletexture are working again when the patch is applied. I also got custom vertex shaders partially working; the problem is I need to figure out how/when to switch back to the Pre Z vertex shader after using a custom one. I have also been playing with GPU PerfStudio and APITrace to try and get some more info about the GPU. So far, I don’t have any new optimizations that give any noticeable results, but I believe the bottleneck is still in the fragment shaders.

Tracker stats:

New: 0
Closed: 1 (1 by me)
Net Change: -1
Current: 153

For those that are interested, in total, I have closed around 65 bug reports as part of this year’s summer of code.

What do you plan to do next week?
Next Monday is the pencils down date, so I’ll start getting some patches ready to send off to code review.

Are there any problems that will require extra attention and what impact will they have on your proposed schedule?

Nope, things are going smoothly.

Are there any builds available?

There are some on GraphicAll.

Cheers,
Moguri

What did you do this week?

While I fixed a couple of bugs, I spent way more time on optimizing this week. I spent some time with AMD’s GPU PerfStudio 2 on an AMD card. After a lot of profiling and toying around, I found that fragment shaders were really slowing down the Necrosys map. I did some research and decided to try implementing a depth pre-pass/Pre-Z pass to reduce overdraw and the amount of time needed to process fragments by culling them with a depth test. For more information on early depth testing, there is this article from AMD, which I found very helpful. This yielded a 60% increase in the fps of the Necrosys map (going from about 63fps to about 104fps) on my system. I started a thread on Blender Artists to try and collect more data on how this optimization affects other scenes.

I also implemented display lists for shadows, but this gave no noticeable performance difference since the bottleneck was not in transferring vertices.

Tracker stats:

New: 4
Closed: 2 (2 by me)
Net Change: +2
Current: 154

What do you plan to do next week?
Next Monday is the “suggested ‘pencils down’ date,” but I don’t think it affects my project a whole lot this year since most of what I’m doing is cleaning and scrubbing. I have some more test files that I will take a look at for optimization. Maybe I can also look into closing enough reports to get back down to three pages in the tracker.

Are there any problems that will require extra attention and what impact will they have on your proposed schedule?

Nope, things are going smoothly.

Are there any builds available?

There are some on GraphicAll.

Cheers,
Moguri

What did you do this week?

Recently I’ve been profiling the BGE’s VBO code, and I’ve been rather disappointed with it. Even after some cleanup/optimization (including a nice speedup to skinned meshes and other frequently updating meshes), I could not get VBOs as fast as vertex arrays with display lists. I’ve also come to find out that Nvidia has a particularly nice display list compiler, which will make it difficult to get Nvidia cards running faster with VBOs than with display lists. I was, however, hoping to at least get ATI and Intel cards to run faster. However, on those two cards, VBOs are still running slower than vertex arrays alone (no display lists!). I might try to get more gains out of VBOs, but I think I might want to start looking elsewhere.

Taking a look at components, I got the branch to compile again and I improved reloading of components; they now update properties instead of recreating (meaning you don’t lose your settings). While looking at the code, I realized that it had some style issues, so I cleaned it up to better match what I could remember from Blender’s style guide (C-style comments in C code, K&R bracing for loops and ifs).

As for the bug tracker, I managed to close the following bugs this week:

  • Action actuator doesn’t finish playing if frame rate drops {fixed r49349}
  • BGE Vertex deformer optimized method does not work properly {fixed r49371}
  • Character physics type colliding with sensor type {fixed r49373}

I also fixed a couple of bugs that were reported to me outside of the tracker:

  • Performance regression with 2D Filters {fixed r49326}
  • Restrict Animation Updates option not framerate independent {fixed r49732}

Tracker stats:

New: 6
Closed: 3 (3 by me)
Net Change: +3
Current: 152

And we’re back to four pages. :(

What do you plan to do next week?
More optimizing and bug fixing, with more of an emphasis on optimizing.

Are there any problems that will require extra attention and what impact will they have on your proposed schedule?

Nope, things are going smoothly.

Are there any builds available?

There are some on GraphicAll.

Cheers,
Moguri

What did you do this week?

I managed to finally get Nvidia Nsight and started profiling some OpenGL usage. I cleaned up some of the BGE’s OpenGL usage (eliminated some glGet and glIsEnabled calls), which got me a few fps in the Necrosys map. I’m hoping for more gains, but I’m still learning how to best use the tool. I also managed to speed up loading on Dalai’s project by using glGenerateMipmap() instead of gluBuild2DMipmaps() on hardware that supports glGenerateMipmap(). This greatly reduced the delay from when you could start hearing sounds to when you could start seeing the level.

In an effort to start getting things merged into trunk (if I’m lucky for 2.64), I’ve submitted a patch with only my changes to Swiss for code review.

Furthermore, I’ve finally gotten around to recreating the ge_components branch with my component code. I haven’t done much testing with it, but I’d like to start poking around and seeing what needs to be done.

I also managed to close the following bugs this week:

  • Incorrect physics for LibLoaded dupligroups {fixed r49237}
  • SubSurf in BGE suggestion {rejected}
  • Action Actuator in Loop End stops updating the Frame Property after no longer receives positive signal {fixed r49189}
  • Unable to modify KX_LightObject in BGE {fixed r49154}
  • light distance not adressable in GE GLSL mode {fixed r49154}
  • Overlay scene gets transparent when motion blur is enabled {r49128}

Tracker stats:

New: 6
Closed: 6 (6 by me)
Net Change: +0
Current: 149

What do you plan to do next week?
Hopefully I can start getting some reviews from the code review. I will continue trying to make the Necrosys map run faster and fix more bug reports.

Are there any problems that will require extra attention and what impact will they have on your proposed schedule?

Nope, things are going smoothly.

Are there any builds available?

There are some on GraphicAll.

Cheers,
Moguri

What did you do this week?

I started off this week looking at multi-uv bugs to see which ones are actually fixed in Swiss. I’ve verified that bugs #18146 and #17927 are fixed in Swiss. #20281 and #37775 should be solved after I get a bit of clarification on some Blender code.

There are also a couple of bugs about changing light values in realtime not having any graphical effect. These bugs were fixed in Cucumber, so I’m going to see if I can bring that code over to Swiss or trunk.

Other than bugs, I managed to get lib loaded materials to not compile their shaders twice. This gets rid of an error message when using the async option, and it offers a small speed up. I have also gotten my Swiss code into a working copy of trunk to start looking at the possibility of merging with trunk.

Now time for some tracker stats:

New: 3
Closed: 1 (1 by me)
Net Change: +2
Current: 149

What do you plan to do next week?
Once Nvidia get’s their developer site back up, I’d like to try some of their programs to profile the BGE’s OpenGL usage to try and get the Necrosys map to run better. I finally got Dalai’s files working, so I can also start trying to optimize for those. Overall, a scene change spends about two seconds on scene conversion; hopefully I can get it down to around one second or better.  I’ll also see about the possibility of merging some of my Swiss changes into trunk so the multi-uv bug reports can be closed.

Are there any problems that will require extra attention and what impact will they have on your proposed schedule?

Nope, things are going smoothly.

Are there any builds available?

There are some on GraphicAll.

Cheers,
Moguri

What did you do this week?

This week was a rather slow week for me as I was busy with other things and waiting on files. However, I’ve fixed some memory leaks in both trunk and Swiss that I found using the handy Visual Leak Detector. I also wrote up a Game Engine release log for the 2.64 test builds release, and I’ve been monitoring response to the release in this BA thread to try and find any regressions. So far, the only regression I’ve confirmed is one involving DDS/DXT textures and needing to flip the compressed textures. This regression (as well as the reason for the regression and possible fixes) is noted in the release log.

Of the bugs I fixed, one notable one was a regression (found prior to the 2.64 release) caused by the character physics type which made Radar and Near sensors collide with objects. I say this is notable, because I think it was one of the few (if not the only) 2.64 BGE regressions in the tracker. Hopefully this means 2.64 won’t break too many 2.63 games. Another bug worth mentioning is enable/disable rigid body not working with Bullet. Now that I’ve fixed this, I don’t think there are any old features (i.e., Sumo features) laying around that do not work with Bullet.

Now time for some tracker stats:

New: 4
Closed: 11 (8 by me)
Net Change: -7
Current: 147

And we are down to three pages in the tracker!

What do you plan to do next week?
I got a map from the Necrosys guys that they are letting me profile, so I’m going to see what I can do with that. So far, it loads about 30% faster in Swiss. Dalai also has a game/walkthrough for me to profile, but we’re having issues running it on Windows. Once those get resolved, I’ll also profile that.

Are there any problems that will require extra attention and what impact will they have on your proposed schedule?

Nope, things are going smoothly.

Are there any builds available?

There are some on GraphicAll.

Cheers,
Moguri

What did you do this week?

I finally implemented an interface to work with asynchronous lib loading. I settled on a future object that can register callbacks as mentioned in this post on my feedback thread. As for the tracker, I managed to close eleven reports, but five new ones came in. The good news is we are now only four reports away from finally getting back to three pages.

Tracker stats:

New: 5
Closed: 11 (11 by me)
Net Change: -6
Current: 154

4 more to go until we’re back down to three pages!

What do you plan to do next week?
I should be able to get the tracker under three pages this next week. As for what non-bug-hunting task I will work on, I think I will need to consult Dalai. I might work on an actuator for LibLoad, but with the Hive GSoC, it might be better to wait on adding any new logic bricks. I could start looking at cleaning up components, or look into more converter optimizations.

Are there any problems that will require extra attention and what impact will they have on your proposed schedule?

Nope, things are going smoothly.

Are there any builds available?

There are some on GraphicAll.

Cheers,
Moguri

What did you do this week?

I’m still awaiting more feedback on the LibLoad thread to figure out which direction I want to go on the interface to asynchronous LibLoad (callback or future object).

I did manage to get some speed ups in the converter for loading non-power-of-two textures by avoiding scaling the textures to powers of two if the hardware has NPOT support. Non-power-of-two textures took about four times as long to load than power-of-two textures, now the times are about equal. However, I haven’t yet committed the change since I haven’t decided if I want to put it in trunk or Swiss.

As for the tracker, with some help from Daniel, I managed to get all of the open bugs organized into the new categories. In the process, I also managed to close 18 bugs this week. One that might be of interest to people is:

[#23375] texture2D in custom 2D filters can get texture outside of game, resulting in ugliness

Essentially, this means you won’t be grabbing the Blender UI in 2D filters anymore.

Tracker stats:

New: 2
Closed: 18 (18 by me)
Net Change: -16
Current: 162

12 more to go until we’re back down to three pages!

What do you plan to do next week?
I would like to get down to three pages in the tracker, which will require me closing 12 more reports. I will also come to a decision this week on how I want async LibLoad to look and implement it.

Are there any problems that will require extra attention and what impact will they have on your proposed schedule?

Nope, things are starting to go smoothly.

Are there any builds available?

There are some on GraphicAll.

Cheers,
Moguri

Follow

Get every new post delivered to your Inbox.