SetLayeredWindowAttributes Window Transparency

[Shrek]

Trying to make the main window transparent with SetLayeredWindowAttributes but I keep getting error 87 (ERROR_INVALID_PARAMETER/The parameter is incorrect)

User32 [COLOR="#FF0000"]=[/COLOR] Library[COLOR="#FF0000"].[/COLOR]Load([COLOR="#800080"]"user32.dll"[/COLOR])
Kernel32 [COLOR="#FF0000"]=[/COLOR] Library[COLOR="#FF0000"].[/COLOR]Load([COLOR="#800080"]"kernel32.dll"[/COLOR])
 
[COLOR="#0000FF"]local[/COLOR] x [COLOR="#FF0000"]=[/COLOR] User32[COLOR="#FF0000"].[/COLOR]GetWindowLongA(Application[COLOR="#FF0000"].[/COLOR]GetWndHandle()[COLOR="#FF0000"],[/COLOR]-[COLOR="#000000"]20[/COLOR])
[COLOR="#0000FF"]if[/COLOR] Bitwise[COLOR="#FF0000"].[/COLOR]And(x[COLOR="#FF0000"],[/COLOR]0x00080000) [COLOR="#FF0000"]==[/COLOR] 0x00080000 [COLOR="#0000FF"]then[/COLOR]
  [COLOR="#0000FF"]if[/COLOR] User32[COLOR="#FF0000"].[/COLOR]SetLayeredWindowAttributes(Application[COLOR="#FF0000"].[/COLOR]GetWndHandle()[COLOR="#FF0000"],[/COLOR][COLOR="#000000"]0[/COLOR][COLOR="#FF0000"],[/COLOR][COLOR="#000000"]0[/COLOR][COLOR="#FF0000"],[/COLOR]0x00000002) [COLOR="#FF0000"]==[/COLOR] [COLOR="#000000"]0[/COLOR] [COLOR="#0000FF"]then[/COLOR]
   Dialog[COLOR="#FF0000"].[/COLOR]Message([COLOR="#800080"]"Error Code"[/COLOR][COLOR="#FF0000"],[/COLOR]Kernel32[COLOR="#FF0000"].[/COLOR]GetLastError())
  [COLOR="#0000FF"]else[/COLOR]
   Dialog[COLOR="#FF0000"].[/COLOR]Message([COLOR="#800080"]"OK"[/COLOR][COLOR="#FF0000"],[/COLOR][COLOR="#800080"]""[/COLOR])
  [COLOR="#0000FF"]end[/COLOR]  
[COLOR="#0000FF"]else[/COLOR]
 Dialog[COLOR="#FF0000"].[/COLOR]Message([COLOR="#800080"]"Error"[/COLOR][COLOR="#FF0000"],[/COLOR][COLOR="#800080"]"No WS_EX_LAYERED"[/COLOR])
[COLOR="#0000FF"]end[/COLOR]

I dont know if this is to do with the COLORREF 'structure' or not and it should be a pointer so if so the structure is not looking like a structure commonly used so if this is the issue how do I fix it? The example should make the window complete transparent.

Using a random dll to deal with transparency then I can restore transparency no problem with:

User32[COLOR="#FF0000"].[/COLOR]SetLayeredWindowAttributes(Application[COLOR="#FF0000"].[/COLOR]GetWndHandle()[COLOR="#FF0000"],[/COLOR][COLOR="#000000"]0[/COLOR][COLOR="#FF0000"],[/COLOR][COLOR="#000000"]255[/COLOR][COLOR="#FF0000"],[/COLOR]0x00000002)

[Shrek]

Bump......................................

[milano88]

function SetLayeredWindowAttributes(hwnd, Color)
User32 = Library.Load("user32.dll");
local nRet = String.ToNumber(User32.GetWindowLongA(hwnd,-20)) + 524288;
User32.SetWindowLongA(hwnd, -20, nRet);
User32.SetLayeredWindowAttributes(hwnd, Color, 0, 1);
end 

Call this function :

SetLayeredWindowAttributes(Application.GetWndHandle(), Math.RGBToNumber(212, 208, 200)); 

[Shrek]

We need the MemoryEx guru to help................................

[Imagine Programming]

COLOREF is just a RGB color integer, not a byref structure, but simply a value. I think I've found another issue though, it seems that MemoryEx doesn't nicely pass on the bAlpha value as a byte, but as an int. I need to fix it.

[Shrek]

Cool, look forward to a fix :yes

[Imagine Programming]

You could make an attempt at writing an Assembly wrapper for the function call, I'll see what I can do tonight. In the meanwhile, I'll add ARGTYPE casts to MemoryEx which will allow you to cast from Lua number to any other integer / floating point type in a function call.

[Imagine Programming]

Try this:

local load, band, bor, sprintf = Library.Load, Bitwise.And, Bitwise.Or, string.format;

local user32 	= load("user32.dll");
local kernel32 	= load("kernel32.dll");

if(not user32 or not kernel32)then
	error("user32 or kernel32 load issue.");
	os.exit(0);
end

local pSetLayeredWindowAttributes = user32:GetProcAddress_("SetLayeredWindowAttributes");
if(SetLayeredWindowAttributes)then
	error("SetLayeredWindowAttributes not present in user32.dll.");
	os.exit(0);
end

SetLayeredWindowAttributes = ASM.Assemble(sprintf([[
	USE32
	ORG		100h
	
	PUSH 	EBP
	MOV 	EBP, ESP
	
	PUSH	DWORD [EBP + 20]
	PUSH 	BYTE  [EBP + 16]
	PUSH 	DWORD [EBP + 12]
	PUSH 	DWORD [EBP + 8]
	
	CALL 	%u
	
	POP 	EBP
	RETN
]], pSetLayeredWindowAttributes));

if(not SetLayeredWindowAttributes.assembled)then
    local asmError = SetLayeredWindowAttributes:GetError();
    if(asmError)then
        if(asmError.errorCode ~= 0)then 
            Dialog.Message("Assembler error: "..tostring(asmError.errorCode), "Line "..tostring(asmError.errorLine)..": "..asmError.errorMessage.."\r\nCode at line: "..asmError.errorLineString);
        end
    end
end

function set()
	local x = user32.GetWindowLongA(Application.GetWndHandle(),-20)
	if band(x,0x00080000) == 0x00080000 then
	  if SetLayeredWindowAttributes(Application.GetWndHandle(),0,0,0x00000002) == 0 then
	   Dialog.Message("Error Code", kernel32.GetLastError())
	  else
	   Dialog.Message("OK","")
	  end  
	else
	 Dialog.Message("Error","No WS_EX_LAYERED")
	end
end;

[Imagine Programming]

Disregard my last post, the stack system in the library system of MemoryEx seems to work just fine. I was worried that it attempted to push single bytes onto the stack, but it doesn't. The attached example works perfectly fine on my end, could you test it? The code executes two functions in the dialog's On Show event. The first one ensures the layered style and the second one sets the transparency.

[Shrek]

Works just fine :yes

You know this ASM is like a diamond in the rough just waiting dazzle us all, just wish I knew how it worked............

[Imagine Programming]

Works just fine :yes

You know this ASM is like a diamond in the rough just waiting dazzle us all, just wish I knew how it worked............

Once you get to know ASM, it all seems logical. All you need to memorize is a few registers and you can google the instructions aside from the basic ones. As for the stack, it'll take a bit longer to get to know all it's perks and quirks. You should experiment with ASM in MemoryEx ^^

It's such a shame that ASM isn't used as much as before anymore. It could optimize so many operations!

Great to hear it's working :yes

[Shrek]

Hi again, so playing with your code then:

user32.SetLayeredWindowAttributes(Application.GetWndHandle(),0,150,0x00000002)

should just work if the window is WS_EX_LAYERED but the main window is not but in my original code:

if Bitwise.And(x,0x00080000) == 0x00080000 then

it reports as it is as it goes through to the then so am I doing the Bitwise checking wrong?

[Imagine Programming]

If I run the next code in On Show in an empty project, it reports
> 0
> false

as it should.The check is valid, so the window might have the WS_EX_LAYERED style
for some reason. Have you used ensure_layered() from my example code by any chance?

local style = user32.GetWindowLongA(Application.GetWndHandle(), -20);
local state = Bitwise.And(style, 0x00080000);

Dialog.Message("test", tostring(state).."\r\n"..tostring(state == 0x00080000));

[Shrek]

I've tried the ensured layered and it fixes it but it reports still on my code as not being WS_LAYRED but it's no drama setting it so anyway as its working now so that's the main thing and that's another random dll done away with.

[Imagine Programming]

I've tried the ensured layered and it fixes it but it reports still on my code as not being WS_LAYRED but it's no drama setting it so anyway as its working now so that's the main thing and that's another random dll done away with.

Nice work! To check what styles are being set on a window, try WinSpy++ (http://www.catch22.net/software/winspy-17)
With this application, you can investigate a window. Make sure you drag the target to the titlebar of the AMS window.