Friday, June 13, 2008

Pixel Shader Wave Effect - Hit-testing in WPF Effect

In my previous post I wrote about a grayscale effect. That effect only changed the color of the GUI. In this post I will describe an effect that will apply a sine-wave effect on a GUI that will displace the controls. The cool thing is that the controls will still work with user interaction. This is quite simple and it is done by implementing an EffectMapping (GeneralTransform) in the ShaderEffect.
Download sample project here.
Download only WaveEffect.dll here.
You must have .NET Framework 3.5 sp1 or later installed for this to work. You also need DirectX SDK installed to be able to compile the pixel shader effect.

image

If you are interested in how pixel shaders work in WPF I refer to my previous post about the grayscale effect. In this post I will not write much about how pixel shaders work in general, but more about how to interact with a GUI after an effect has been applied. As I mentioned before there are different types of effects. An effect can modify the appearance of a GUI (e.g. grayscale, modify colors or contrast...) or it can apply an effect that will displace the controls from its original positions. The wave effect used in this example is an displacement effect that will apply a simple sine-wave to the GUI. Download the sample project above for source code.

1. The Sine-Wave Algorithm
The algorithm is quite simple and we will use it both in out shader file (HLSL-file) and also in our transform class. This is the math part:
Output.Y = Input.Y + (Math.Sin(Input.X * Frequency) * Amplitude);
Output.X = Input.X;
What is this? Ok, we will have some input, a x-value and a y-value, and then provide some output, a x-value and y-value. Think of them as coordinates in an image. In this example we only modify the y value and therefore the output x will be the same as the input x. We then calculate the y-value by taking the input y and add another value to it. The added value comes from a Sine-calculation meaning that it will be a +/- value. This also means that for a position y we will instead output a position above or below the input value, this will create the wave effect. In the Sine-calculation we have two properties that we can change to modify the appearance of the wave (amplitude and frequency).

2. WaveEffect : ShaderEffect
This is the C#-effect that we use in WPF and XAML. It it the interface toward the pixel shader and the GPU. The difference between this effect and the grayscale effect is that we have overridden a property called EffectMapping:

private WaveTransform _waveTransform = new WaveTransform();
protected override GeneralTransform EffectMapping
{
get
{
return _waveTransform;
}
}




We provide a customized transform class that inherits from GeneralTransform.



3. WaveTransform : GeneralTransform

This is out transform class that is associated with our pixel shader (WaveEffect.fx). When we apply the shader, we will displace the controls from their original positions. If we then try to mouse click the control where we see it on screen, we might actually miss is because in the logic of our program it is placed at a different location. The EffectMapping is here to help us. If we implement this mapping of positions, the framework can ask the transform class how the displayed position actually maps to the logical position. It is actually very simple, if the user clicks at screen position x,y, the framework asks the transform-class: -Hey, where is this position in my logical world?


Note that we can not create a pixel shader that is created from total random numbers (noise) and at the same time apply interaction logic. In other words, there are a very strong relation between the pixel shader and the transform. First look at our pixel shader:



sampler2D implicitInput : register(s0);
float frequency : register(c0);
float amplitude : register(c1);

float4 main(float2 uv : TEXCOORD) : COLOR
{
uv.y
= uv.y + (sin(uv.x * frequency) * amplitude);
float4 color
= tex2D(implicitInput, uv);

return color;
}




And this algorithm corresponds to the algorithm in the following WaveTransform (see TryTransform, TransformBounds):

public class WaveTransform : GeneralTransform, ICloneable
{
public WaveTransform()
{ }

public static readonly DependencyProperty FrequencyProperty = DependencyProperty.Register("Frequency", typeof(double), typeof(WaveTransform), new PropertyMetadata(1.0), new ValidateValueCallback(IsValidFrequency));
public double Frequency
{
get { return (double)GetValue(FrequencyProperty); }
set { SetValue(FrequencyProperty, value); }
}

public static bool IsValidFrequency(object value)
{
double checkValue = (double)value;

if (checkValue < 0.0)
{
return false;
}

return true;
}

public static readonly DependencyProperty AmplitudeProperty = DependencyProperty.Register("Amplitude", typeof(double), typeof(WaveTransform), new PropertyMetadata(0.2), new ValidateValueCallback(IsValidAmplitude));
public double Amplitude
{
get { return (double)GetValue(AmplitudeProperty); }
set { SetValue(AmplitudeProperty, value); }
}

public static bool IsValidAmplitude(object value)
{
double checkValue = (double)value;

if (checkValue < 0.0)
{
return false;
}

return true;
}

private bool _isInverse = false;
private bool IsInverse
{
get { return _isInverse; }
set { _isInverse = value; }
}

public override GeneralTransform Inverse
{
get
{
WaveTransform newWaveTransform
= (WaveTransform)this.Clone();
newWaveTransform.IsInverse
= !IsInverse;
return newWaveTransform;
}
}

/// <summary>
/// Transform a bounding box to/from Sin-wave.
/// </summary>
/// <param name="rect"></param>
/// <returns></returns>
public override Rect TransformBounds(Rect rect)
{
Rect boundingBox
= rect;

if (IsInverse)
{
boundingBox.Y
= rect.Y - Amplitude;
boundingBox.Height
= rect.Height + (2 * Amplitude);
}
else
{
boundingBox.Y
= rect.Y + Amplitude;
boundingBox.Height
= rect.Height - (2 * Amplitude);
}
return boundingBox;
}

/// <summary>
/// Transform a point to/from the Sin-wave. Use Inverse to get the inverse transformation
/// of the Sine-wave.
/// </summary>
/// <param name="inPoint"></param>
/// <param name="result"></param>
/// <returns></returns>
public override bool TryTransform(Point inPoint, out Point result)
{
result
= new Point();

if (IsInverse)
{
result.Y
= inPoint.Y + (Math.Sin(inPoint.X * Frequency) * Amplitude);
result.X
= inPoint.X;
}
else
{
result.Y
= inPoint.Y - (Math.Sin(inPoint.X * Frequency) * Amplitude);
result.X
= inPoint.X;
}
return true;
}

protected override Freezable CreateInstanceCore()
{
return new WaveTransform();
}

protected override void CloneCore(Freezable sourceFreezable)
{
WaveTransform newWaveTransform
= (WaveTransform)sourceFreezable;
base.CloneCore(newWaveTransform);
this.CopyCommon(newWaveTransform);
}

#region ICloneable Members

public new object Clone()
{
WaveTransform newWaveTransform
= new WaveTransform();
newWaveTransform.CopyCommon(
this);

return newWaveTransform;
}

#endregion ICloneable Members

private void CopyCommon(WaveTransform transform)
{
this.IsInverse = transform.IsInverse;
this.Frequency = transform.Frequency;
this.Amplitude = transform.Amplitude;
}
}




This is pretty much it. Now we have an effect that displaces the controls (WaveEffect.fx and WaveEffect.cs), but we also have a transform mapping (WaveTransform.cs) that makes it possible to interact with the displaced controls. How are we going to use this functionality? Well, I don't know yet, I haven't found a very good area for this, however... It is really cool!!! :)
I think we can use this for a maginfying glass/zoom effect and maybe for some focus-context stuff. In the final release of .NET 3.5 sp1, there will be some added functionality to this. One question that I have is if the pixel shader only works for Pixel Shader version 2.0 (ps_2_0). I tried to implement a mandelbrot effect but it failed due to the shader version. Does anyone know about this? I would like it to work with ps_3_0!

5 comments:

James Hurst said...

Hi - excellent article. It's the first to shed some light on a DirectX WPF Effect that involves translation along a spatial axis. Something I've been puzzling over.

I was wondering - suppose we wanted to simply create a smoothly scrolling (horizontally) image, such as a waveform (that's been converted to a bitmap Image)? How do we know how many times /second the code gets called (I guess that would be called the frame rate?), or how to control that frequency?

I appreciate your taking your time to illuminate these issues for us.

James Hurst

Anders Bursjöö said...

Hi James,
I do not know if I understand your question? For example if you want your image/border/some other control to be scrolled and at the same time have an effect applied to it, then you just need to use a ScrollViewer around that element and everything will just work :). I tried this with the sample code and it works fine, quite cool whitch a wave-scrollbar.

If your question is how often these effects get updated, then I think the answer is in each renderpass or actually each time WPF rerenders itself. If you change somthing in the layout (for example, you are scrolling a control) then the effect will also be updated.

If you would like more of an animation then I think you can just add an animation to your control and it will work.

Did this answer your question?

James Hurst said...

Thanks for your reply Anders. WPF doesn't allow me to animate the scroll-position of a ScrollViewer. So I tried a timer at various framerates, giving the ScrollViewer a nudge at each timer-tick, and regardless of the rate it shows a visible horizontal jitter. And I'd like to use less CPU time. So, I want to try a DirectX Effect, whereby perhaps just every 250ms I tell the content of the display (which has 32 complex waveforms, rendered into a bitmap) to move horizontally by some small amount over the next 250ms, so as to give it a perfectly smooth, horizontal scrolling affect.
So, in your opinion - what method should we call, to cause the image to re-render itself. I'm thinking that I should be able to just, say at a 60Hz timer rate, set a parameter to the DirectX HLSL code to tell it how far to move, and signal it to do the move (in a smoother, less cpu-intensive way than the ScrollViewer would, hopefully?).

Incidentally, I'm unable to build your Wave Effect project. Using VS 2008 SP1, it's saying "The tag WaveEffect does not exist in XML namespace.." Is there something I'm missing?

Thank you!

JH

Chris Hagan said...

Just to answer the previous commenter's question for posterity:

If you're getting the 'tag does not exist' it's a knock on error. The main issue is that the WaveEffect project did not compile. This might be that fxc.exe is (dependent on? supplied by?) the DirectX SDK, which is a slim 520mb download. Once this is in place the project will compile.

Sharath said...

Hi,
I have a document page view for which I am applying this grayscale effect.
It works fine, its just that I see a small shift in the glyphs (text) when the effect is applied.
Not able to figure out why its happening. Any thoughts?