The Infinite Staircase

Cinesite built an integral part of the interior of Hogwarts, the great marble staircase where a large portion of the action, battles in particular, takes place. It was built as a very large CG asset that could appear in many scenes through out the film at different times of day, and also be destroyed. CG digital supervisor Holger Voss said, “It was developed from a big set piece built at Pinewood Studios. This staircase has appeared in the earlier films, sometimes with rotating steps, but in this film it becomes much larger, extending into the distance within a very large environment.”

The set piece extended about 1 and 1/2 stories, each way, and appeared dressed according to the sequence. Although large, it actually only represented a small section of the staircase. “We took a pretty standard approach, collected all textures, took a Lidar scan and total station survey and built a module that we could re-use and replicate. At the time we re-built it we still didn’t know the level of destruction, type of lighting or the size of the environment, so we tried to be prepared.”

The principle photography for ‘Deathly Hallows Part 2’ was done back to back with ‘Part 1’, and because the sequences had already been shot when the team started, the practical set had been destroyed and built up again. They had some blueprints showing that it needed to essentially extend into infinity. From these, Cinesite developed the looks and produced basic renders to submit for approval.

Perfect Lighting

Harry and Hermione

The production was often shooting right off the set and into the rafters, because the set was so large that they couldn’t put a green screen above it when the action approached the top of the stairs although the actors might be running back up into the lighting rigs. The team would then have to rotoscope the actors and change lighting set ups. Other challenges arose in the cut. In one shot you might see a large area of the practical set behind an actor, just before cutting to a close up of the same actor with a green screen behind him. The lighting would have to match perfectly and the staircase build needed to accommodate close up viewing.

On set the team spent a full day measuring and photographing this set. Unfortunately, because the plates had already been shot, they couldn’t record scene lighting and instead re-lit their work to match the plate. In fact the lighting had to be adjusted and art directed anyway, for example to show natural light variations in certain areas wherever the stairs were seen receding into the distant background. Also, because the staircase was such a complex construction with so many arches and doorways, they usually chose to light it with global illumination in Renderman to avoid complications with too many light sources.Flexible Compositors

Flexible Compositors

“Our compositing team was small, only 13, relative to the number of shots. The lead compositor and lighter developed tools allowing them to share set ups for compositing the staircase,” said Andy Robinson, 2D digital supervisor. “Within each set up were sliders for haze levels, haze colours, defocus and so on. The settings could be saved as a key set up for a scene and loaded into the same node in another shot. They knew that they would have to be working intensely toward the end of production so they created these set ups early, tweaking them whenever they had time. Such measures helped improve our consistency.”

Cinesite’s artists have made the animated portraits in all the Potter films and it has become the specialty of one of their compositors, Karen Wand. The technique, updated this time to work in Nuke, involves a 2D approach, starting with motion analysis on the character and using brush strokes, canvas and varnish textures found on paintings with colour correction to make the colours more muted.

“We have some proprietary motion vector analysis tools, and using this motion analysis data we stick the textures to the characters as they move through the painting to make the canvas itself appear to move. A physical painting is supplied for the background into which the characters are composited. When Ariana walks back into the painting, they shrunk her down slightly,” said Andy.

One shot involves 15 portraits on a wall in the marble staircase environment. Production shot various scenes of characters fleeing a set piece, and then choreographed the same characters to flee through the set piece of what would be the portrait next to it on the wall. Editorial carefully pieced the vignettes together as a line-up sheet for the team to match.

Head Tracking

Voldemort

Cinesite had been responsible for replacing actor Ralph Fiennes’ nose area with Lord Voldemort’s snake-like snout in ‘Deathly Hallows Part 1’, and carried on with the task. They completed about 130 such shots, contrasting with only about 50 in the previous one. The same nose asset was used, but with a much larger team to track and light the shots. Andy said, “The changes made for this film were made to the process, not so much to the asset, shader or textures. We were trying to keep the soft tracks with the same artist wherever possible, to keep the facial expressions very similar across all shots within each scene.”

Lead compositor Martin Ciastko and his team joined forces with the lighting and tracking teams to decide how accurate the 3D had to be, shot by shot. As one of the film’s lead characters, Voldemort’s looks and performance were critical to the story. Sometimes the actor’s performance demanded very precise match move work to capture enough subtlety but at other times the model could be loaded directly in the composite in Nuke and the textures re-projected.

Skin Shader

To give the director a clear idea of Voldemort’s presence in each scene early on, the team had to make a good start on the replacements across the film. “First, we tracked the camera to the background, then tracked the overall movement of Voldemort’s head according to that camera. Once this movement was approved, we could start on the facial match move,” said Holger. This also meant the basic lighting could be completed and a preliminary element given to the compositors, who cleaned up the tracking markers and live action nose, and produced an initial version of the shot.

The build of Voldemort’s head was based on a Cyberscan of the actor, and the geometry was used to generate a detailed displacement, showing pores and wrinkles. To help preserve the integrity of his performance and achieve an accurate match move, the team built a rig with three levels of animation controls, accessing the full set of tracking markers attached to his head with enough flexibility to match the action in each sequence.

They had developed a skin shader tool to generate the snake nose, generating textures by using specialised photography and lenses, and extracted detailed pore maps. Multilevel subsurface scattering improved the photorealism. Some shot-specific dirt maps and dirt textures were also added where Voldemort picks up dust and dirt during the fights.

Cinesite’s doe patronus this time was created with the same technique as before – rendered using an in-house fur pipeline with interior swirling effects through a custom volume shader using Renderman – but its behaviour was to be quite different. Instead of slowly moving through a dark forest scene, referring to Harry’s mother, it appeared as Snape’s patronus and leapt like a rabbit as it did in ‘The Order of the Phoenix’, and they used both as their reference. New swirling ribbon effects were added in Houdini as a particle simulation converted to geometry.

Green Gunfight

When the team was asked to develop some entirely new wand effects, they put their matte painters and 2D artists to work on concepts. They created some still frames and then multiple frames dissolved together to get a feel for a new effect. They applied several wand effects in a scene the included a chain wrapping around a character, the deflected bolt that blows up part of the staircase, and Ron’s spell fired at an opponent with a flowing ink affect. All of these were a chance to be creative.

For Harry’s and Voldemort’s critical battle on the staircase, Cinesite created the first portion of an effect that MPC finished with a ribbon effect. This was handled like a gunfight with green bolts that Harry deflects, all done in 2D. “We designed and executed the first part of their wand fight using Nuke, trying to use the green of the Avada Kedavra spell and incorporating some explosive shield effects as Harry tries to defend himself. Harry deflects the wand duel effect into the staircase ceiling. This allowed for one of our biggest effects in the film, where our staircase completely crashes down onto Voldemort.

Wrapped in Chains

“We used some practical flame and water elements as a basis and heavily warped and treated them in Nuke, mapping them onto spheres which formed the shield around Harry and making them feel explosive with bright bursts of light which augmented practical interactive lighting on set,” Andy said. “In contrast, the wand fight between Snape and McGonagall, briefed as a fiery effect, a whip or a flame thrower, was 3D. Each of these has to be planned separately, usually starting off in 2D for a quick turnaround for look development cycles for the director, and we may decide to get into 3D later for effects such as fire.

For this fight in Hogwart’s Great Hall, simple animation rigs were used to drive the particle effect, instead of fluids alone. They chose particles with curl noise fields to achieve a fluid look with less computation time and also re-lit the environment with glow and flashes from Snape’s shield to integrate it into the Hall.

The actor, a Deatheater, who was wrapped in chains was shot reacting to something, as if he had been hit, giving them an opportunity to try various concepts – a boulder, a cloud of dust. His movements required a match move to help animate the chains coming from Harry’s wand. Since there was a lot of cloth movement in the character, they also assisted the match move in 2D for a more fluid approach to stick the chains to the character’s body. “‘Harry Potter’ has such a strong visual sense. That’s what makes it satisfying to be asked to use this as a starting point to come up with something new.

Rubble and Debris

At that early stage, it was important to build the staircase to be able to achieve as much destruction as was called for in any plate, revealed by how much rubble and debris had been included, for example. For speed and flexibility, they built it in real pieces, every brick and part, and wrote some software so they could paint in the destruction and then run a simple rigid body simulation to drop the rubble onto the staircase.

They had investigated various rigid body solvers to destroy the staircase with, and chose PullDownIt from Thinkinetic in Spain, the group that wrote the first solver for RealFlow. “Cinesite bought a site license for it early on and we extended its functionality to do some set dressing by painting onto the steps where you want to apply certain types of debris or rubble. The solver would drop this material onto the steps accordingly, so that nothing was intersecting or floating, in nice believable piles using physical parameters of gravity, mass and centre of mass,” said Holger.

This process was carried out per scene, depending on plates and the story. 2D Digital Supervisor Andy Robinson said, “When Harry Potter and Voldemort are fighting, Harry deflects a big wand effect up into the staircase and rubble comes crashing down. We never had a definitive shot for it, just a couple of wide shots, plus a medium shot of Harry cowering under a shower of debris. So we came up with some dramatic camera angles and interesting lighting to use in production, backlit and dark, as look development for the destruction in the sequence.

“The shot didn’t get official approval but was cut into the film early enough to build on. The freedom to do this comes from having created such a large, flexible CG asset. We could work with whatever the plates presented.” Holger also built some generic 3D camera angles to take out on second unit shoots of extra characters – such as people fighting, wizards, kids and death eaters – to composite into the backgrounds of the staircase shots.