ZephyrBurst wrote:I'm assuming dxtory gets its data from the video driver and then does... something? That would be why there was the hall of mirrors effect in the Pete room. It's one of maybe 2 maps in the game that isn't using some sort of image for a background and instead is filling the background with a color.
Dxtory, as well as Fraps, and OBS in "game capture" mode (and I'm sure some others), all work by dynamically inserting a hook into the game's code, such that
right before the directx or opengl call it makes to flip the back/front buffers, it will run a directx/opengl call to read the frame from video memory to shared memory that the capture program reads out of. That capture itself isn't what likely interfered with things, that's pretty harmless normally. This method of inserting a hook in the game's code that catches when it flips the front/back buffer is nice because it allows the capture software the match the frame rate of the game perfectly. Other screen capture methods (i.e. OBS in "window capture" mode) that don't do it this way, can't synchronize to the game engine's frame rate so things might not be quite as smooth/consistent.
In order to do an overlay of capture status, some of these types of software will also use this *same* hook to render their own HUD ontop of everything else using some directx/opengl calls. This rendering of the capture software's HUD during the capture hook is what messed things up. It seems that dxtory's HUD rendering code didn't clean up after itself very well, and left things in a state that is incompatible with Game Maker's method of rendering a color as background at the start of the next frame. This consequently causes the background to not be redrawn there, both on screen and in capture.
Also, the HUD showing in the recording during screen transitions, likely means that Game Maker was flipping the front/back buffers during those screen transitions without actually clearing/redrawing the framebuffer, which is a bit of an odd engine behaviour though perfectly valid. Dxtory and such software avoids capturing it's own HUD by only drawing the HUD after it captured the frame it's drawing the HUD onto... but... if the engine doesn't redraw anything and keeps flipping frames, the capture software will go and capture it's own HUD when the next frame comes around.
Random note, Steam Overlay, and most VOIP software in-game overlays, all work very similarly to this as well by inserting a hook in the running game's code too in order to render themselves. These also present similar potential issues if they fail to clean up after themselves well when rendering their overlay.