Getting around Blend Mode limitations of Forms/WPF compositing

Apr 19, 2011 at 1:28 PM
Edited Apr 22, 2011 at 11:30 PM

This proposes a shell concept and a code example for indirectly manipulating and altering the underlying blend functionality, from afar, to produce other blend effects.  The focus of this example in particular is to illustate how to achive Additive type blend functionality and derrivatives using the underlying Alpha blend composite functionality already built in and hopfully and presumably executing in hardware.

This is really a hack to get around the lack of blend mode support in .Net 4.0 itself. There is no alternative support to the singular Alpha Blend which is compositing safe anywhere in the managed world unless you look to XNA, which is impractical to use within VS.  In trying to emulate the rendering of WoW for its textures for AddOn Studio, something was needed to render using other blend modes, as some textures require blend modes other than the "Alpha Blend".

What exists now:

Windows GDI+ and by extension Windows Forms use a lowest common denominator implementation of a classic compositing scheme which most modern graphical operating systems use.  Many new OSs also implement a relatively common Blend Mode based on canonical Alpha Blend algo., where basically source and destination components are altered by one or more Alpha values, where the source alpha value strengthens the source compenents and weakens the destination components which are then added together to create the blend.  So Alpha Blend Mode is a weighted blend based on an Alpha factor.  Additive Blend Mode is the same except the blend factor Alpha is not negated on the destination component.  The historical reasons for using an Additive Blend (and why applications like flash and wow use it) are: because its cheaper computationally, less data needs to be used potentially, and because the canonical Alpha Blend does not cover all effects by far.  A multicolor highlight and all effects that derive from that basic technique, for example, would be impossible with Alpha Blend, even with multiple passes, without destroying the underlying background.

An illustration of the blend mode "Hack":

Below are screen shots from an unreleased version of AddOn Studio, not from WoW. First with the new functionality disabled, and then with it enabled on the second image below. The technique of pushing the highlight texture past the margins of its text is used all over WoW on every list, as a way to equalize the effect of the highlight across the text, knowing that the blend mode was supposed to be additive, and you get a nice very subtle highlight on the text borders for free. 

 Blending via "Alpha Blend":

Alpha Blend

 Blending via "Additive Blend":

Additive Blend

Other than not having implemented the brighness correction yet on the Additive Blend version, the main difference you will see is in how the underlying mechanics of the blends work.  In these examples the "black" parts of the texture will always get mixed with the background with an Alpha Blend, where as with the Additive they will not.  Additive will "Add" its source colors to the underlying destination background, which Alpha will never be able to do by itself.  We are hacking the underlying Blend function that we dont have access to, to act like Additive Blend, by hyjacking the textures Alpha channel in an attempt to get the final math on what numbers each pixel become, to be as though an Additive function had been run instead.  With text under the highlight the Alpha Blend would be even worse, as the text would get clobbered in similar manner to the border, rather than being highlighted.  And by clobbered I mean that the background text will get weakened by whatever the inverse of the Alpha "transparency" constant is during the blend, thus it will "look" like the text is under the "shadow" of the highlight rather than above or a part of it, or illumiated by it.

Brightness correction:

 Blending via "Additive Blend" - FullBright with input Alpha Scale by 0.75:

Additive Blend - FullBright

30 mins later... I've now added "FullBright" where the color components in the texture are forced to 255 and with the Alpha following Red texture component, and Scaling the input color Alpha property by 0.75f. Notice how the darkness of the fade still counteracts the Alpha Blend function, still pretending to be Additive.  So rgba of 30,30,30,0(implied 1) becomes 255,255,255,30 and farther near black the effective input alpha of say 4,4,4,0(1) becomes 255,255,255,(4/0.75=)1 , while the pattern in the texture is now held purely in the alpha component.

Notice how even with high quality modes turned off, you still see both the pattern in the highlight and the patten in the background, with the illusion of an additive highlight rather than a regular transparency Alpha.  Becuase this texture was not greyscale (the underlying texture is whiteish with a color matrix applied before draw with glowing yellow color), and because the 0.75 Alpha scale in the color matrix, the texture has lost part of its firey glow, and part of the color matching that followed the button text color. This can be brought back by scanning the texure for its extents and either adding the max val difference from 255 to all components or using the difference to scale and if you want to be super fancy can query for gamma and apply that too, pre draw.  However the point of this was to not go there at all, really, as we are tryign to *not* do it all in software.  There are plenty of examples of doing application side compositing all in software, and our goals are to try to do as little as possible and honor both possible hardware rendering and the mechanics of the compositing system.  So instead, to counter-act the effects cheaply, you can back off the input Alpha scale or not use it, and simply scale the fullbright back some, to give the Color Matrix a little more room to wiggle the applied color from the texture property.   Also I have discovered that scaling the input Alpha value on the ColorMatrix side is "out" because it washes the colors in the .Net implementation more than I thought it would.   So.. the fastest computational path wiht the best results is: let image component Alpha folow any component, add 126 in the bottom of the color matrix and dont scale the input Alpha.  However if your render path uses the color matrix to color the texture for Draw, you will wash your color transform out as well by usinghte bottom row for a correction constant.  In that case you will have to control the brightness correction on altering the "fullbright" level or by scaling the ColorMatrix colors before adding them to the matrix.

Final results:

And two hours later...  lol

 Blending via "Alpha Blend" - With Text using 0.7 Alpha:

Alpha Blend - 0.7 Alpha

 Blending via "Additive Blend" - With Text:

Additive Blend - With Text and 4/5 Bright

 Blending via "Additive Blend" - WoW In-Game:

Additive Blend - WoW in Game

BTW, 10k gold to whoever can guess what the name of the addon is... :)

My Additive Blend hack still lacks the impact of the WoW rendering, though in fairness its a larger scale and im still doing only 4/5 Fullbright.  All three of these are pixel for pixel scale streight from print screen.  I think its close enough, and I know the font rendering isnt right with the outline and its rendering native 12pt instead of 10pt like wow is because addon studio is correctly scaling hte font two steps larger for the larger coord scale.  Theres work to be done in quite a few other places, but FrameXML Designer starting to get scarry close overall.  The most important thing is that it works.  Its effectivly Additive Blend mode without having to resort to a full draw it myself, so the blends are hopefully in hardware with extremely minor transforms on the source texture.  I will say the pipe setup, at least for scale and blending for .Net is no where near as good as WoW, especially when I turned on the "high quality" settings for Graphics (which are all turned off in AddOn Studio for these screen shots).  The higher quality settings are "no bueno" for this type of "compositing".  

If you compare the last "Alpha Blend" vs the "Additive Blend" shots, where all I did was change 'Blend' to 'Add' on the 'Alpha Blend' Property then change the Alpha from 0.7 to 1, you can see a very good example of why I added this Blend Mode.  If you loaded an existing AddOn set correctly expecting Add (which woudl ahve had its alpha at 1.0) then you would see no text at all, as Add dosent require additional alpha scaling just to do its blend job (which is why it is normally also screaming fast). 

Soap Box "On":

Reguardless of all this, I believe this "hack" might open some doors for others currently wondering why everything else seems to have modern rendering options and then getting frustrated :)  I mean no FUD, bashing or ill sentiment towards windows core paths for drawing.  You have Flash and friends, HTML 5.0 and freinds, all sorts of phones, which leads into XNA which requires a differnt runtime, you have Silverlight which is way off in another direction, and thats pretty much it unless you want to try to play backbuffer city with either hand rendering / PixelShaders or trying to COM youself to death with DX in the CLR.  This is all to say it should be possible for someone to do a few basic things in the core path, reguardless of the historical occlusion based compositing paradgm and its very necessary place in windows, which keeps the faithful at some lower level of parity wiht the flash, small devices, and 5.0s of the world.  WPF is wonderful except it excludes a few key facits that shouldnt be religated jsut to SL and that one market.  The more i work on this the more the stark contrast hits me, and is why I'm writing this soap box blurb.  I hope someoneis workgn on this.  BTW, the font rendering is still fuzzy in the editor window, just kidding (no really).

The code concept:

Whats happening in the code example below is that the Red component is being used as a darkness factor, and the Alpha component is being altered in an attempt to negate the Alpha Blend negative effect on the destination components.  This creates the *partial* illusion for the purposes of AddOn Studio of an Additive Blend.  What woudl be nice is if there was a second canned blend mode besides alpha exposed though windows all the way up through GDI+.  I think most would want Add Blend.  Iv'e read the arguments abotu compositing and WPF and the whole "how many back buffers will this take" kinda thing.  However I'd say if its possible to do Alpha, which is noting more than a formula buried down in some ancient Win32 GDI driver/hardware functionality, then its possible to do Add.  I dont know why things are the way they are exactly, but im sure there some reason.  I'm also pretty sure there is Add already down in ther some where, since its the basis for Alpha.

If you try to use this be aware that you will probably need to also use a ColorMatrix with an alpha other than 1 on Draw to balance the effect.  As this stands, in taking a raw image and trying to normalize the Alpha component on the image only, it will be opaque still where ever its bright, and not a true additive blend.  If you know that your texture/image is monochrome/grayscale, you can simply "fullbright" the texture by setting all components other than alpha to 255 and then apply a ColorMatrix Alpha of 0.5f on the Draw and you will end up wiht someting very close to a true additive blend.  You could also set the color components to 127 and allow the originally intended Alpha setting to be used in the ColorMatrix, and process through the blend, which would be the most complete solution. 

So:

- if you have a texture that is not monochrome:  use this and set the ColorMatrix alpha to 0.5f and play with either manual brightening by altering hte texture compoenents directly, or alter the additive or multiplicitive color values in the color matrix to get your brightness back.

- if you have a monochrome texture and no incoming Alpha factor to set on color matrix: use this and fullbright the texture, then set 0.5f+ on the alpha in the color matrix.

- if you have a monochrome texture and *do* have an incoming Alpha factor to set on color matrix:  use this, and set teh color compoents to 126 (halfbright) manually and let your incoming alpha factor pass through the color matrix unaltered.  Or, full bright and scale the incomming alpha like: incomming alpha (desired transparency) * 0.5f. 

You will still have to play around with gettign the bright ness back as all of htese techniques qill morethan likely be darker than what woudl have happened if you had Additive Blend.

 Ill have this out in todays (4/19/2011) release of AddOn Studio if you want to play wiht it in action:

http://www.wowwiki.com/AddOn_Studio_2010

The code:

This is a crude unoptimized example, with parts liberated from the MSDN Documentation LockBits code example simply as an illustration.  The Code in AddOn Studio 2010 is a good bit different but basically the same idea.

namespace Microsoft.WowAddonStudio.FrameXml.Formats.Convert
{
	public struct AquireImage
	{
		public Image image;
		public bool hasAquired;

		public Bitmap bitmap;
		public BitmapData bitmapData;
		public IntPtr ptr;
		public int bytes;
		public byte[] rgbValues;

		public AquireImage(Image image)
		{
			this = new AquireImage();

			this.image = image;
			if ((image.PixelFormat & PixelFormat.Canonical) == 0)
				return;
	
			// Obtain copy and lock
			bitmap = new Bitmap((Image)image.Clone());
			bitmapData = bitmap.LockBits(
				new Rectangle(0, 0, bitmap.Width, bitmap.Height),
				ImageLockMode.ReadWrite, bitmap.PixelFormat);

			// Get the address of the first line.
			bytes = Math.Abs(bitmapData.Stride) * bitmap.Height;
			rgbValues = new byte[bytes];

			// Copy the RGB values into the array.
			ptr = bitmapData.Scan0;
			System.Runtime.InteropServices.Marshal.Copy(ptr, rgbValues, 0, bytes);

			hasAquired = true;
		}

		public Image ReleaseImage()
		{
			if (!hasAquired)
				return null;

			var bitmap = this.bitmap;

			// Copy the RGB values back to the bitmap
			System.Runtime.InteropServices.Marshal.Copy(rgbValues, 0, ptr, bytes);

			// Unlock the bits.
			bitmap.UnlockBits(bitmapData);

			this = new AquireImage();

			return bitmap;
		}
	}

	public static class BlendModes
	{
		public static Image ConvertToAddMode(Image image)
		{
			var bitmap = new AquireImage(image);
			if (!bitmap.hasAquired)
				return image;

 			var p = bitmap.rgbValues;
			for (int i = 3; i < p.Length; i += 4)
				p[i] = p[i-1];

			return bitmap.ReleaseImage();
		}
	}
}