Nearly there!

So, it's past the end of the month, and is PyIgnition ready for the 1.0 release? Well, actually, the code is. All the new features are already implemented and theoretically bugless. However, the documentation (which is fairly important seeing as the only purpose of the project is to be used by others) lags significantly behind the numerous ruthless (read: careless and ill-planned) changes I've made to the way the library works since the last release. Not to mention that it's lacking any information whatsoever on all the new features. But that's in progress at the moment, and with any luck it should be a very brief process leading to a prompt release!

But why the delay? Well, the file loading function ended up being slightly more complicated than anticipated. All was going perfectly well until the XML parser I was using decided that it wasn't going to read one of the tags in the test file, for no good reason whatsoever. At the same time, I was noticing occasional bugs in its attempts at reading tag meta data (meta data being variables specified within the tag itself, eg. <tagname meta="value" meta2="another value">). Which of course meant that it would arbitrarily fail to read certain files. Attempts at delving into the gungy mess of ASCII-encoded vomit that passed for the XML parser's source code quickly met with horrors so horrifically horrifying that I would be accused of torture were I to write about them here, but to sum up, I'm extremely surprised it ever worked at all.

So, I had to write a new XML parser. Much in the way of fun and frolics ensued, I can tell you. However, the end product is actually reasonably good (certainly a lot more so than the parser I was using beforehand). The source, in case you're interested, can be viewed and downloaded from here. And now PyIgnition can save and load files!

Now, on a more positive note than my abject failure to meet any deadline which I set for myself, I'm pleased to tell you that development work on Obsidian (the particle effects creation/editing program for PyIgnition) is progressing nicely. PyIgnition embeds in wxPython perfectly and is currently running rather smoothly. The design of the actual application is well underway, and I have here the first mockup - it's of the particle editor window which lets you edit and keyframe particles for a particular source:

(Click for a full-size version.) I've even got a functional demo of the keyframe timeline widget pictured, which had to be written from scratch as such a thing doesn't exist in wxPython (funnily enough):

The slider is draggable and, after a lot of messy code, clicking on keyframe markers takes you to their frames. It also sends out wx.EVT_SLIDER events when the slider is moved (whether by dragging or clicking on a keyframe marker). How many lines for this tiny, minimalistic GUI control? Over 170. Aah, the joys of wxPython. But I digress. All is currently well on the Obsidian front, and with any luck the mocked-up window shall soon transcend its current SVG state and become a Real Window!

On an unrelated side-note, Scottish exam results go out on Thursday - and we the cool kids who signed up for the online day-before text-message-and-email service will know tomorrow! 'Tis a truly terrifying state of affairs indeed. If they are absolutely flawless (As everywhere), I'll be off to Cambridge in October; otherwise I'll be going to Edinburgh. The former isn't looking particularly likely at the moment, but well. I shall nevertheless know tomorrow what'll be happening. Not in the morning, of course; that would be too nice. The results actually go out at 2:50 in the afternoon for reasons beyond my comprehension, so tomorrow morning will almost certainly yield the most painful lazy lie-in I've ever lived through. Glorious! Anyway, that's all for now; good day to you all!

Almost a month on

And where are we? Well, this time it hasn't been so much a case of laziness or time-wastery that's led to such a huge gap between posts: I'm pleased to announce that PyIgnition is still faring very well. The 1.0 release is still going very much to plan, and will most likely be finished by the end of this month. To share news on all that has been added since the last update here would take far too long so I'll instead give you this link to the page for the 1.0 release, which gives a comprehensive list of everything that has been or is still to be done, and then elaborate upon the more interesting developments here.

Behold! A screenshot!

What's this mad spiffery, you say? Vortex gravity! Yep, I managed to get it to work. It was surprisingly difficult actually, and it's still not yet perfect (firing particles directly at it causes all sorts of unintended madness), but it's at least working reasonably well. Essentially it calculates how far a given particle is from its centre, and then using that along with the magnitude of the particle's velocity it determines the centripetal force required to keep it perfectly in orbit. It then adds to that a tiny additional force (also towards its centre) so that the particle will accelerate inwards whilst rotating around it. Of course, it uses a slight variation on the process outlined here as calculating the distance and the magnitude of the velocity for every particle would involve a lot of square roots (and they're evil, as we know), but the force calculated is the same. And that's vortex gravity!

Another key development is 'cushion distance' for particles which have radii. Previously, collisions with obstacles would use particle centres alone which, although fine for points, meant that circular particles would appear to go partway through obstacles before colliding. Getting collisions to use these particles' radii was a long and arduous process, but the results are a significant improvement - and deserving of another screenshot!

You'll notice that there's still a bit of overlap in the 'after' image - this is entirely intentional. Without any overlap whatsoever, the gaps between particles show up very clearly and look a bit strange, and sometimes numerical inaccuracies lead to particles hovering just above obstacles. So, I've added a parameter to the 'constants.py' file (which is another new addition, incidentally) which specifies what factor of the radius is allowed to pass through obstacles. At the moment, this is set to 0.3 (30% of the radius), which seems to work quite well.

The last (and most significant of all, but sadly lacking in pretty screenshots) development I'll be writing about on this fine morning is the PyIgnition Particle Effect (PPE) file format. It's an XML-based file format for storing ParticleEffect objects in their entirety (keyframes and all) such that they can be easily re-loaded into another ParticleEffect later. At present the format's design is complete and the function for writing to it is finished - see here for a sample output. Reading from it is a bit more tricky, but progress has been good: it can already read particle sources and all three gravity objects (including all keyframes and particle keyframes), which leaves only the obstacle objects unreadable at present. This is the last feature on the list for the 1.0 release, so as mentioned at the beginning of the post I'm fairly certain the planned release date of the 30th should be a meetable one.

Updates to follow, ideally close behind!

Keyframe madness

Behold, the promised post! It promises promising developments, developments which seek to fulfil the promise of a promising new keyframe system. There we go, now the word 'promise' looks strange to me. If it doesn't to you, try re-reading the last two sentences again. It's a pretty silly word, isn't it? But I digress.

You may recall from the last post that I've been working on a brand new keyframing system for PyIgnition. Indeed, the more astute amongst you may recall from the above paragraph that developments of a promising nature have occurred in the aforementioned area. Well, I'm glad to say that both observations are indeed in line with the most glorious reality about to unfold before you. That is, the new keyframe system is fully implemented and working rather well.

However, the coding process most certainly hasn't been without challenge. Indeed, it was rather horrific. Essentially, the new keyframe system introduced a somewhat more complicated InterpolateKeyFrames() function which took a little bit longer to complete. Shouldn't really be a problem, no? Well, when this function is running every frame on several hundred particles, each with several keyframes and variables to loop through, the slight difference becomes a seriously massive difference. As a result, even the simplest of scripts like the original fire demo would grind to a near-halt within seconds. And unfortunately no amount of ridiculous over-optimisation of the interpolation function yielded any real improvement.

But wait! All is not lost! For once all hope seemed to have dropped away (much like PyIgnition's frame rates), a guiding light of truth shone through the muck of malformed code and illuminated a speedy solution, bright and gleaming, the Excalibur of my code. Well, to be fair it wasn't quite as romanticised as that. In reality I spent many an age trawling the web for better ways to interpolate between frames, whilst simultaneously trying every optimisation technique I could find on the existing code - literally to the point of trying to use local variables over global variables because they're apparently faster, and avoiding using object.function() (preferring a simple function() ) because it's supposedly slower. The latter saw no avail, regrettably, but the former took me on a most interesting tour of the interwebs. And, I'm pleased to say, I came upon a solution!

You see, the particles belonging to one source are all spawned with exactly the same keyframes - so the same interpolation calculations are being done for every particle, just at different times. This is obviously wasteful. So instead of doing these same calculations, every single time, the program now pre-calculates what the variables of a particle should be for every frame of its life and saves them all to a big old lookup-table-esque array thing. This is stored in the particle source object, and is accessed from created particles through a reference to their parent source. When a particle keyframe is created with the source via the CreateParticleKeyframe() method, the source immediately updates this lookup-table-esque array using its PreCalculateParticles() function.

You'll probably have noticed that there's a definite space-time tradeoff here. Obviously the particle keyframes no longer pose any problems whatsoever, as the particle update method now simply sets each of the particle's variables to the ones stored by the parent source for the current frame. This means that the program is now actually slightly faster than it was before the new system was implemented. However, the thought of storing every variable for every frame of a particle's life is slightly unnerving, as one would assume that it would mean massive memory consumption. However, this is actually not as significant as you might think.

Here's an example. We have a fairly typical particle effect with keyframed particles, which emits 20 particles per frame with a particle lifespan of 600 (both of these are probably larger than you'd ever need to use in practice, but I'm using exceptionally large values just to show how little memory the particle caching actually requires). Now, at present, a particle object can keyframe the following variables:

• colour - 3*int = 12 bytes
  
• radius - 1*float = 8 bytes
  
• length - 1*float = 8 bytes

This gives us a total size of 28 bytes for a single set of stored variables. Now, I haven't the foggiest idea how Python stores dictionary objects (which is how these variables are stored in the lookup-table-esque array thing), so just for laughs we'll double this value to 56 bytes, a size it will probably not come close to reaching in reality. Again, this is just to show how little space it actually requires. So, to store keyframed values for every frame of a particle's life, we would need to store 600 of these variable sets. That gives us a final memory usage of 33, 600 bytes or 33.6 kilobytes. Not bad for an overly large value you're unlikely to ever reach in practice, eh? So really, all things considered, this business of pre-calculating keyframed particles is almost certainly the way forward. It's definitely made things run faster anyway.

And that's where I am now. Well, actually, I also added setter methods for keyframeable variables - simply setting them in the 'object.variable = newvalue' fashion simply won't work now as it would be overridden by keyframe interpolation, so you would now use 'object.SetVariable(newvalue)', which creates a keyframe for 'variable' at the supplied value on the current frame. Next up is giving the user the opportunity to select interpolation modes (followed by a bit of bugfixing of course), and the next release - which has been upgraded to Beta 1, incidentally - will be ready!

...And the post frequency meter was striking thirteen!

Yes, I know - two posts in as many days! Utter madness! The post frequency meter is indeed frazzled. But this is merely a swift news-deposit before I head off to undertake the elaborate job of implementing the new keyframing system in PyIgnition for the Alpha 4 ('controlled eruption') release. Incidentally, on that topic, I've just this morning sorted out completely how I'm going to implement the new system. It's all scribbled down on assorted scraps of paper (many of which lie within my own head), and just needs to be coded now. However, as mentioned a few sentences ago, it'll be a long and difficult process. Updates will follow when it's underway!

So, to return to the planned purpose of this present posting, I've done a bit of dabbling in wxPython today for the first time in ages and I must say I'm addicted all over again. It's simply brilliant. And, I'm pleased to announce, fully compatible with Pygame after a little bit of fiddling around. Here for your viewing is a screenshot of 'Wind.py' running on a wxPanel, completely under the command of wxPython:


The window is even resizeable, and the particle effect display scales with it. Incidentally, 'Obsidian' is a name that we were originally considering for what is now PyIgnition, and at the moment is probably going to be the name of the particle effects editor. More incidentally, the more astute amongst you may have noticed that the window looks a bit different to those seen in screenshots I've posted in the past. Why, you ask? Well, I've finally upgraded from the old XP machine (which is probably getting near its sixth birthday now), and with the new one came a new OS: Windows 7. It's rather good actually; I fully agree with those saying it's a vast improvement over Vista. But I digress.

Using pygame in wxPython applications is actually not too complicated a process, although it was incredibly difficult to find anything about it on the Internet when I looked so I'll post my findings here in the hope that the next person who tries it will find them through a Google search. There's only really one source I could find that explains it, and that's the one at http://wiki.wxpython.org/IntegratingPyGame. However, it's a bit iffy in its explanation. In case you don't notice immediately, they miss the part where they actually give the wxFrame a child to use as a pygame display - you'll need to remember to create this yourself. I just used a simple wxPanel.

The code to look at is line 10 - this makes SDL see your widget (or wxWindow, to be terminologically accurate) as the display. From there you can simply use pygame.display.init() as shown, and get a handle to the screen with self.screen = pygame.display.set_mode(). Normally in Pygame one would supply screen dimensions to that function, but here it's probably a good idea to leave them out. However I haven't tried supplying them so I don't know whether it would work or not, so feel free to experiment and see what happens. Now, although the link says you shouldn't import pygame until after setting the SDL_WINDOWID parameter, this is actually a pain because you then don't have access to pygame's functions anywhere else. I actually found that it works perfectly well if you do import pygame at the top of the file, so there's probably nothing wrong with doing so any more. That said, I'm just going with what seems to work so feel free to correct me if I'm wrong and there are hidden issues with doing so.

With that done, you're free to do things like self.screen.fill((0, 0, 0)) and pygame.draw.aaline(screen, [blah...]) to draw to the pygame surface, so all is well from there on. But is it? As a matter of fact, you'll face problems when you try to exit - neither pygame nor wxPython wants to close first, and you'll get errors either way. However, after a bit of messing around I've managed to find a solution to this. Simply bind the following function to WX_EVT_CLOSE:

____def Kill(self, event):
________self.Unbind(event = wx.EVT_PAINT, handler = [your drawing function])
________pygame.quit()
________self.Destroy()

See, the problem that arose in making Pygame quit first was that wxPython would for some reason try to draw again before it exited. This naturally causes problems when your drawing function contains Pygame draw functions, as they will be operating on a surface which has been disabled on Pygame's exit. Unbinding the draw function avoids this issue, and lets pygame exit first with wxPython smoothly following suit afterwards.

And that's pretty much it. So far I haven't found any more issues with the two working together, but I'll update if I do. I hope this proved somewhat helpful. So, back to keyframes!
Goodbye!

It was a bright cold day in April...

You asked me once, what was in Post 101. I told you that you knew the answer already. Everyone knows it. The thing that is in Post 101 is... well, pretty cool. All cultural references aside, the content of this amusingly-numbered post is in fact rather spiffy, and comes in the manifestation of an update on PyIgnition's progress. To summarise, it's been going rather spiffily. So spiffiliy indeed that I decided it warranted a post. So! To make haste t'wards the very tip of this post's point, I shall cast in your general direction some video footage of PyIgnition, post mutatio:

PyIgnition alpha 3 from Animatinator on Vimeo.

So as you can see, the promised physics system is very much in place. Particles can now deflect off objects thanks to a combination of what is effectively an inverse gravity (but using an inverse cube law instead of inverse square to ensure objects aren't visibly repelled from a distance), and a preceding intersection resolution stage. The latter is a fairly simple process which firstly works out whether any particles are inside obstacles and then shifts them back out to appropriate places. The two procedures are defined as functions for collision obstacles (as InsideObject() and GetResolved() respectively), so the code handling the overall process is just a couple of lines:

if (not obstacle.OutOfRange(particle.pos)) and (obstacle.InsideObject(particle.pos)):
____particle.pos = obstacle.GetResolved(particle.pos)

(The first part of the if statement is just checking if the particle is within reasonable range of the obstacle before it checks for an intersection.) I also added a 'bounce' multiplier to the repulsive force which is mainly set to a value below 1.0, as the force often tends to be a bit big and results in particles gently drifting towards an obstacle only to be propelled away so quickly that the program would need to take the effects of special relativity into account in order to process them correctly. In all of the demos shown this was set to a standard value of 0.2, which causes only a slight bounce when particles collide with obstacles.

So, that concludes this decade's blog update. See you next time, when I'll hopefully be bringing news about the work-in-progress new-and-vastly-improved keyframe system!