index > Windows Forms General > detect user inactivity (idle, timeout)

detect user inactivity (idle, timeout)

I know this question has been addressed recently but bear with me.

My application (vb.net 2005) consists of severalforms in a hierarchy that present

info to the user. Form1 calls form2 (showdialog) which calls form3 etc...

I want the forms to timeout and return to form1 if the user walks away (ok, its a kiosk app)

Each form has its own timer(system.win.forms.timer). When this matures the click

event closes preceding forms and then itself, returning the system to form1.

Some forms consist of nothing but labels and text boxes. I can easily detect activity

in the text boxes and reset the timer (timer.stop,timer.start seems to do the trick)

But form3 presents a PDF within a webbrowser control. I can't detect activity

within this control. I've tried the following :-

1. form.keypreview / form.keypress

2. webbrowser.navigating, does nothing once doc is loaded

3. form implements ImessageFilter does not report movement within webbrowser

4. API SetWindowsHookEx again does not work within the PDF

The last method I got from http://support.microsoft.com/Default.aspx?id=319524

In the last paragraph it says "You cannot implement global hooks in Microsoft .NET Framework"

Is this the crux of the problem ???

Looking forward to your thoughts.

sovande Tuesday, April 24, 2007 8:34 AM

Thanks to serendipity (and nobugz) I think the problem is solved.

Apparently its important to turn off the runtime debug host when doing this low level hooking

otherwise the hook attempt fails.

When searching on WH_MOUSE_LL I found some code by nobugz to intercept mouse position.

With help from another post (Ryan Tsai 10-30-2006) I figured out how to detect mouse button up/down events(WM_LBUTTONDOWN)

So now when the user is happily scrolling through a PDF within an activex control my app knows about

it and won't timeout.

I do like to know about the internals on computers but when you've got a deadline to meet now hasn't

been the best time to get into interop.

Thanks for the help, and here's the code (mostly from nobugz)

Imports System.Runtime.InteropServices

Public Class Form1

Private Structure MSLLHOOKSTRUCT

Public pt As Point

Public mouseData As Int32

Public flags As Int32

Public time As Int32

Public extra As IntPtr

End Structure

Private Const WM_LBUTTONDOWN As Int32 = &H201

Private Const WM_LBUTTONUP As Int32 = &H202

Private _mouseHook As IntPtr

Private Const WH_MOUSE_LL As Int32 = 14

Private Delegate Function CallBack(ByVal nCode As Int32, ByVal wParam As IntPtr, ByRef lParam As MSLLHOOKSTRUCT) As Int32

<MarshalAs(UnmanagedType.FunctionPtr)> Private _mouseProc As CallBack

Private Declare Function SetWindowsHookExW Lib "user32.dll" (ByVal idHook As Int32, ByVal HookProc As CallBack, ByVal hInstance As IntPtr, ByVal wParam As Int32) As IntPtr

Private Declare Function UnhookWindowsHookEx Lib "user32.dll" (ByVal hook As IntPtr) As Boolean

Private Declare Function CallNextHookEx Lib "user32.dll" (ByVal idHook As Int32, ByVal nCode As Int32, ByVal wParam As IntPtr, ByRef lParam As MSLLHOOKSTRUCT) As Int32

Private Declare Function GetCurrentThreadId Lib "kernel32.dll" () As Integer

Private Declare Function GetModuleHandleW Lib "kernel32.dll" (ByVal fakezero As IntPtr) As IntPtr

Public Function InstallHook() As Boolean

If _mouseHook = IntPtr.Zero Then

_mouseProc = New CallBack(AddressOf MouseHookProc)

_mouseHook = SetWindowsHookExW(WH_MOUSE_LL, _mouseProc, GetModuleHandleW(IntPtr.Zero), 0)

End If

Return _mouseHook <> IntPtr.Zero

End Function

Public Sub RemoveHook()

If _mouseHook = IntPtr.Zero Then Return

UnhookWindowsHookEx(_mouseHook)

_mouseHook = IntPtr.Zero

End Sub

Private Shared Function MouseHookProc(ByVal nCode As Int32, ByVal wParam As IntPtr, ByRef lParam As MSLLHOOKSTRUCT) As Int32

'Debug.Print("Message = {0}, x={1}, y={2}", wParam.ToInt32, lParam.pt.X, lParam.pt.Y)

'Form1.Text = CStr(lParam.pt.X) & " " & CStr(lParam.pt.Y)

If wParam = WM_LBUTTONDOWN Then

Form1.TextBox1.Text &= CStr(wParam) & " "

End If

Return CallNextHookEx(WH_MOUSE_LL, nCode, wParam, lParam)

End Function

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

'Application.AddMessageFilter(Me)

Me.AxAcroPDF1.LoadFile("c:\test.pdf")

Me.WebBrowser1.Navigate("c:\ALurem\Backup Solution\Internal\Pdf1.pdf")

End Sub

Private Sub btnHook_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnHook.Click

InstallHook()

End Sub

Private Sub btnUnhook_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnUnhook.Click

RemoveHook()

End Sub

End Class

sovande Wednesday, April 25, 2007 11:47 AM

Possibly, AcroRd32.exe starts when you load a PDF document in a browser. Try using Spy++ to find out what process owns the window handle. As an alternative, you could use the Acrobat Reader ActiveX control to display the document so you don't need the browser (and its tendency to leak memory like a sieve). Use it by adding a reference to acropdf.dll, available in the c:\program files\adobe\acrobat 7.0\activex folder.

nobugz Tuesday, April 24, 2007 10:27 AM

Thanks for the quick response.

I'll try acrobat activex again, but I'mpretty sureits the same behaviour for all activex controls.

I wasn't aware that the webbrowser control leaked memory. I was hoping it would be more

reliable than acrobat. As a side note, I've been surprised how many third party PDF tools are out

there to create/manage/view PDF docs. Maybe this is a sign of how flaky acrobat is ?

But sticking to the issue at hand I'll look into spy++ (another new think to learn ;-)

There was another web example of hooking into keyboard/mouse:

http://www.developer.com/net/net/article.php/11087_2193301_3

I couldn't get this to work, the hook failed. I don't know if this an issue with the example code

or if I'm not using the code right. At this stage I don't know enough low level stuff but I think

that's about to change.

Thanks again

sovande Tuesday, April 24, 2007 11:33 AM

OK, So I found Spy++ in the c++ tools and found that the PDF and WEB activex controls

on a form have a PID corresponding to the Adobe AcroRD32.exe, which makes sense.

After another hour searching the internet I find there are three types of hook:

A system hook allows you to insert a callback function which intercepts certain Windows messages (e.g., mouse related messages).
A local system hook is a system hook that is called only when the specified messages are processed by a single thread.
A global system hook is a system hook that is called when the specified messages are processed by any application on the entire system.

I guess I either want a global hook, or a way to find the PID of the background acroRD32 and monitor that.

sovande Tuesday, April 24, 2007 2:56 PM

You, you'd need a global hook to spy on the window messages. No, you can't make one with .NET, it requires a DLL that can be injected into AcroRd32.exe. You can however create a WH_KEYBOARD_LL and a WH_MOUSE_LL hook. While you can't make those specific for the window you want to look at, you could assume that seeing no activity on these hook callbacks indicates the user walked away.

nobugz Tuesday, April 24, 2007 3:12 PM

Do you actually need to detect activity in each specific window? Or, are you just interested in inactivity of the entiresession (e.g. no keyboard or mouse input for 60 seconds)?

If checking the entire session is okay, take a look at the GetLastInputInfo call in user32. You could call this every second or so from a timer, and as abonus,that will occupy less resources than using hooks.

cds_ks Tuesday, April 24, 2007 3:43 PM

The application is for a kiosk. Each form is 1280.1024 borderless so covers the entire screen.

There shouldn't be anything significant running in the background.

No activity in keyboard (virtual) or mouse (its a touch screen) would indicate user has walked away.

Of course it was easyto detect activity in textbox controls using the given events. I can

then reset the form timer with me.timertop,me.timer.start. (by the way I had to guess this

wayofresetting the timer as there is no reset method as far as I can tell from the documentation)

Its the activex control (whether its a webbrower or adobe control) that causes the problem.

The user could be happily scrolling through the document and the app wouldn't know a thing

about it and so timeout.

With my limited experience in using spy++ (about 10 minutes!) I did notice that some activity

was being logged when I clicked on the activex scrollbars:

WM_LBUTTONDOWN, WM_MOUSEACTIVE and WM_PARENTNOTIFY.

I wonder if these correspond to the events mentioned by the above poster ?

Time to investigate further.

sovande Tuesday, April 24, 2007 5:23 PM

I'm constantly amazed how difficult it is to find information.

OK, I know VS is a huge product, but I've just spent 20 mins

trying to find the value of WH_MOUSE_LL in MSDN and failed.

I finally found a value in someones code project after searching for

a related value. Apparently WH_MOUSE_LL = 14, maybe.

I would rather see this in black and white in the documentation somwhere.

I've been considering purchasing a MSDN subscription but if its more

of what I've already experienced I don't think I'll bother.

M (in time to stop and have a beer mode)

sovande Tuesday, April 24, 2007 5:55 PM

You need to look in the Platform SDK header files (winuser.h, mostly) to find values like that. "cds_ks" suggestion is a good one, visit www.pinvoke.net for the P/Invoke declaration. Cheers.

nobugz Tuesday, April 24, 2007 6:19 PM

Thanks to serendipity (and nobugz) I think the problem is solved.

Apparently its important to turn off the runtime debug host when doing this low level hooking

otherwise the hook attempt fails.

When searching on WH_MOUSE_LL I found some code by nobugz to intercept mouse position.

With help from another post (Ryan Tsai 10-30-2006) I figured out how to detect mouse button up/down events(WM_LBUTTONDOWN)

So now when the user is happily scrolling through a PDF within an activex control my app knows about

it and won't timeout.

I do like to know about the internals on computers but when you've got a deadline to meet now hasn't

been the best time to get into interop.

Thanks for the help, and here's the code (mostly from nobugz)

Imports System.Runtime.InteropServices

Public Class Form1

Private Structure MSLLHOOKSTRUCT

Public pt As Point

Public mouseData As Int32

Public flags As Int32

Public time As Int32

Public extra As IntPtr

End Structure

Private Const WM_LBUTTONDOWN As Int32 = &H201

Private Const WM_LBUTTONUP As Int32 = &H202

Private _mouseHook As IntPtr

Private Const WH_MOUSE_LL As Int32 = 14

Private Delegate Function CallBack(ByVal nCode As Int32, ByVal wParam As IntPtr, ByRef lParam As MSLLHOOKSTRUCT) As Int32

<MarshalAs(UnmanagedType.FunctionPtr)> Private _mouseProc As CallBack

Private Declare Function SetWindowsHookExW Lib "user32.dll" (ByVal idHook As Int32, ByVal HookProc As CallBack, ByVal hInstance As IntPtr, ByVal wParam As Int32) As IntPtr

Private Declare Function UnhookWindowsHookEx Lib "user32.dll" (ByVal hook As IntPtr) As Boolean

Private Declare Function CallNextHookEx Lib "user32.dll" (ByVal idHook As Int32, ByVal nCode As Int32, ByVal wParam As IntPtr, ByRef lParam As MSLLHOOKSTRUCT) As Int32