Hello,

I have received a infrared multi-touch frame (Nexio HID Multi-Touch ATI0320-10) where my boss asked me to check if I can make a program that count the number of times I interact with the frame (everytime an object enters the grid). I need to display the amount of times I interacted with the frame as well as the location of my hand/the object. I am a computer science engineer student and this is way out of my field but I said I will give it a try. I do not have problems with writing code and I can use python, c, c++, java and c#. However, I do now know how to even interact with the frame. I do not know what type of data it sends and receive or how to communicate with it at all. I do not even know where to begin to be honest even after searching for a while. As a last resort maybe someone can guide me what to read upon/how to start. The frame is connected to the computer through usb cable and works like a mouse when interacting with it. I guess I just need to be able to read the data it send to the computer inorder to maybe write a program. Any ideas how I can start? The frame looks like this. It does not come with any drivers at all or any sdk files or any information at all. Please help!

Thanks in advance

 

How familiar are you with the Windows operating system?
You need to access the location of the buffer that holds the X and Y coordinates of the mouse. You can then use these values to monitor any interaction.
I am not a Windows guru. Maybe someone with more Windows programming experience can help you find it.

 

Hello,

I have received a infrared multi-touch frame (Nexio HID Multi-Touch ATI0320-10) where my boss asked me to check if I can make a program that count the number of times I interact with the frame (everytime an object enters the grid). I need to display the amount of times I interacted with the frame as well as the location of my hand/the object. I am a computer science engineer student and this is way out of my field but I said I will give it a try. I do not have problems with writing code and I can use python, c, c++, java and c#. However, I do now know how to even interact with the frame. I do not know what type of data it sends and receive or how to communicate with it at all. I do not even know where to begin to be honest even after searching for a while. As a last resort maybe someone can guide me what to read upon/how to start. The frame is connected to the computer through usb cable and works like a mouse when interacting with it. I guess I just need to be able to read the data it send to the computer inorder to maybe write a program. Any ideas how I can start? The frame looks like this. It does not come with any drivers at all or any sdk files or any information at all. Please help!

Thanks in advance

Where to begin is to download a copy of the relevant USB specification. Then you need to dive into the appropriate drivers provided by the manufacturer. Short of those two things being in your knowledge base, I can't see how you can accomplish your task.

What I can tell you is that ALL USB devices begin by having an exchange of information with the PC called "enumeration". The PC makes a database of attached USB devices which have globally unique identifiers. It knows when devices are unplugged, then plugged back in, or switched out for other devices. People do not normally write code on a PC that interacts with a device directly, they always go through an OS API called a driver. Windows would probably not let you talk directly to the device under any conceivable circumstances that I am aware of.

One more thing. I strongly recommend that you purchase a USB data monitor so you can see what is going in and out of the device without having to depend on either the device or Windows. They can range in price from a couple of hundred dollars all the way up to the 5-10K range.

 

Did your boss give you a budget to use for this project?
There is a graphical programming language called "VEE Pro". It was developed by Hewlett Packard Test and Measurement group in 1991 to make it easier to communicate with electronic test equipment to develop automatic test systems. It is now sold by Keysight Technologies. It can be used to communicate with devices on GPIB, serial bus and USB. It may be gross overkill for what you need but it is the best solution I can suggest.
It has a fairly steep learning curve because it has very comprehensive reading, controlling and displaying abilities but it is very intuitive. Programming is done by constructing a multilevel flow diagram of your process on the screen. This is compiled and becomes your program.
It is available as a free trial download to allow you to see if it will do what you want. The limitation is that you can only save your programs if you purchase a license.
I have used it ever since it was first available to write programs to control many different complex automatic test systems.
It may be worth downloading a free copy to play with, to see if you can access the data from the multi-touch frame.

 

For Linux you would do something like this to read HID events. Pretty simple. There must be a simple Windows equivalent.
Event1 is the mouse on my system.

https://www.programmersought.com/article/55905398846/

#include <stdio.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <errno.h>
#include <linux/input.h>
int main( void )
{
        int                     fd;
        fd_set                  rds;
        int                     ret;
        struct input_event      event;
        struct timeval          time;
        fd = open( "/dev/input/event1", O_RDONLY );
        if ( fd < 0 )
        {
                perror( "/dev/input/event1" );
                return(-1);
        }
        while ( 1 )
        {
                FD_ZERO( &rds );
                FD_SET( fd, &rds );
                 /*Call select to check whether data can be read from the /dev/input/event1 device*/
                ret = select( fd + 1, &rds, NULL, NULL, NULL );
                if ( ret < 0 )
                {
                        perror( "select" );
                        return(-1);
                }
                 /*Can read data*/
                else if ( FD_ISSET( fd, &rds ) )
                {
                        ret     = read( fd, &event, sizeof(struct input_event) );
                        time    = event.time;
                        printf( "timeS=%d,timeUS=%d,type=%d,code=%d,value=%d\n", time.tv_sec, time.tv_usec, event.type, event.code, event.value );
                }
        }
         /*Close the device file handle*/
        close( fd );
        return(0);
}

Mouse data.

 

How familiar are you with the Windows operating system?
You need to access the location of the buffer that holds the X and Y coordinates of the mouse. You can then use these values to monitor any interaction.
I am not a Windows guru. Maybe someone with more Windows programming experience can help you find it.

I did think of this approach, this will allow me to get the x and y coordinates without needing to access multi-touch device itself. Mabey I will take this approach if no other options will be available.

 

Where to begin is to download a copy of the relevant USB specification. Then you need to dive into the appropriate drivers provided by the manufacturer. Short of those two things being in your knowledge base, I can't see how you can accomplish your task.

What I can tell you is that ALL USB devices begin by having an exchange of information with the PC called "enumeration". The PC makes a database of attached USB devices which have globally unique identifiers. It knows when devices are unplugged, then plugged back in, or switched out for other devices. People do not normally write code on a PC that interacts with a device directly, they always go through an OS API called a driver. Windows would probably not let you talk directly to the device under any conceivable circumstances that I am aware of.

One more thing. I strongly recommend that you purchase a USB data monitor so you can see what is going in and out of the device without having to depend on either the device or Windows. They can range in price from a couple of hundred dollars all the way up to the 5-10K range.

Thanks for the information. I do not think my boss spend any money on the project. He just want me to check if I can program it without buying anything expensive, in this case couple of hundreds to 5-10k is too much. I will try to understand how I will proceed further.

 

Did your boss give you a budget to use for this project?
There is a graphical programming language called "VEE Pro". It was developed by Hewlett Packard Test and Measurement group in 1991 to make it easier to communicate with electronic test equipment to develop automatic test systems. It is now sold by Keysight Technologies. It can be used to communicate with devices on GPIB, serial bus and USB. It may be gross overkill for what you need but it is the best solution I can suggest.
It has a fairly steep learning curve because it has very comprehensive reading, controlling and displaying abilities but it is very intuitive. Programming is done by constructing a multilevel flow diagram of your process on the screen. This is compiled and becomes your program.
It is available as a free trial download to allow you to see if it will do what you want. The limitation is that you can only save your programs if you purchase a license.
I have used it ever since it was first available to write programs to control many different complex automatic test systems.
It may be worth downloading a free copy to play with, to see if you can access the data from the multi-touch frame.

Thanks, I will try it out!

 

For Linux you would do something like this to read HID events. Pretty simple. There must be a simple Windows equivalent.
Event1 is the mouse on my system.

https://www.programmersought.com/article/55905398846/

#include <stdio.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <errno.h>
#include <linux/input.h>
int main( void )
{
        int                     fd;
        fd_set                  rds;
        int                     ret;
        struct input_event      event;
        struct timeval          time;
        fd = open( "/dev/input/event1", O_RDONLY );
        if ( fd < 0 )
        {
                perror( "/dev/input/event1" );
                return(-1);
        }
        while ( 1 )
        {
                FD_ZERO( &rds );
                FD_SET( fd, &rds );
                 /*Call select to check whether data can be read from the /dev/input/event1 device*/
                ret = select( fd + 1, &rds, NULL, NULL, NULL );
                if ( ret < 0 )
                {
                        perror( "select" );
                        return(-1);
                }
                 /*Can read data*/
                else if ( FD_ISSET( fd, &rds ) )
                {
                        ret     = read( fd, &event, sizeof(struct input_event) );
                        time    = event.time;
                        printf( "timeS=%d,timeUS=%d,type=%d,code=%d,value=%d\n", time.tv_sec, time.tv_usec, event.type, event.code, event.value );
                }
        }
         /*Close the device file handle*/
        close( fd );
        return(0);
}

Mouse data.

Thanks, I will try this out tomorrow and tell you how it went

 

Where to begin is to download a copy of the relevant USB specification. Then you need to dive into the appropriate drivers provided by the manufacturer. Short of those two things being in your knowledge base, I can't see how you can accomplish your task.

What I can tell you is that ALL USB devices begin by having an exchange of information with the PC called "enumeration". The PC makes a database of attached USB devices which have globally unique identifiers. It knows when devices are unplugged, then plugged back in, or switched out for other devices. People do not normally write code on a PC that interacts with a device directly, they always go through an OS API called a driver. Windows would probably not let you talk directly to the device under any conceivable circumstances that I am aware of.

One more thing. I strongly recommend that you purchase a USB data monitor so you can see what is going in and out of the device without having to depend on either the device or Windows. They can range in price from a couple of hundred dollars all the way up to the 5-10K range.

I do not think I need to communicate with the device itself, now that I am thinking of it. The only thing I need is to be able to read the data that the device sends to the computer. For example I am pretty sure that data about X and Y coordinates is sent from the device to the computer so the computer can change mouse position. If I can somehow read this data I can just make a program that listens to when data is sent from device to computer. I make a counter and everytime it receives data I increase the counter by one and also retrieve the X and Y coordinates sent and show them on the screen. Maybe something like this is possible? Then I do not directly communicate with the device but just read the data.

 

For Linux you would do something like this to read HID events. Pretty simple. There must be a simple Windows equivalent.
Event1 is the mouse on my system.

https://www.programmersought.com/article/55905398846/

#include <stdio.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <errno.h>
#include <linux/input.h>
int main( void )
{
        int                     fd;
        fd_set                  rds;
        int                     ret;
        struct input_event      event;
        struct timeval          time;
        fd = open( "/dev/input/event1", O_RDONLY );
        if ( fd < 0 )
        {
                perror( "/dev/input/event1" );
                return(-1);
        }
        while ( 1 )
        {
                FD_ZERO( &rds );
                FD_SET( fd, &rds );
                 /*Call select to check whether data can be read from the /dev/input/event1 device*/
                ret = select( fd + 1, &rds, NULL, NULL, NULL );
                if ( ret < 0 )
                {
                        perror( "select" );
                        return(-1);
                }
                 /*Can read data*/
                else if ( FD_ISSET( fd, &rds ) )
                {
                        ret     = read( fd, &event, sizeof(struct input_event) );
                        time    = event.time;
                        printf( "timeS=%d,timeUS=%d,type=%d,code=%d,value=%d\n", time.tv_sec, time.tv_usec, event.type, event.code, event.value );
                }
        }
         /*Close the device file handle*/
        close( fd );
        return(0);
}

Mouse data.

I am not really sure how to use this code. Please help me understand it. Where do I run it and how do I know it will communicate with my device (shouldn't I provide vendor id and product id to the code? or how can I make it work. Thanks in advance!

 

I am not really sure how to use this code. Please help me understand it. Where do I run it and how do I know it will communicate with my device (shouldn't I provide vendor id and product id to the code? or how can I make it work. Thanks in advance!

If you are asking these types of questions then it's time to tell your boss this problem is beyond your capabilities.

 

I do not think I need to communicate with the device itself, now that I am thinking of it. The only thing I need is to be able to read the data that the device sends to the computer. For example I am pretty sure that data about X and Y coordinates is sent from the device to the computer so the computer can change mouse position. If I can somehow read this data I can just make a program that listens to when data is sent from device to computer. I make a counter and everytime it receives data I increase the counter by one and also retrieve the X and Y coordinates sent and show them on the screen. Maybe something like this is possible? Then I do not directly communicate with the device but just read the data.

I think this task is far more challenging than either you or your boss think it is. It is nice to see people with such confidence when it comes to things they know very little about. May you always find the easiest solution with the least amount of effort.

 

If you are asking these types of questions then it's time to tell your boss this problem is beyond your capabilities.

That is putting it just about as gently as possible in the circumstances. I might add that rooting around in an unfamiliar piece of software looking for something that is not familiar to you is not a task with a high probability of success.

 

I think this task is far more challenging than either you or your boss think it is. It is nice to see people with such confidence when it comes to things they know very little about. May you always find the easiest solution with the least amount of effort.

I understand, I will try to read more... need to come with something asap

 

If you are asking these types of questions then it's time to tell your boss this problem is beyond your capabilities.

I have already mentioned this is not my field thats why I ask hehe, I know I am way out of my field but trying to get an understanding anyway

 

What OS will this be using?

I have both linux and windows, I can one whatever is best for this task

 

Have a look here: http://www.developphp.com/video/JavaScript/Read-Mouse-Coordinates-Pointer-Position-Tutorial

Will do! Thanks.

 

A quick search gave me this https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/usb-event-tracing-for-windows. It looks like a good starting point.

Thanks @nsaspook for your code. I have always used Wireshark for USB mainly while sorting out PIC USB programming. I'll have to venture down this road for kicks sometime.

Edit...

After a little more looking the link won't work for what I want, but it did show some things that will come in handy.