Getting Started with xethru-python-driver

Discussion in 'Presence Sensor' started by Lovexethru, Aug 10, 2017.

  1. Lovexethru

    Lovexethru New Member

    I'm looking for simple presence detection. I'm on Windows and connected X4M300 via USB. I executed the presence_example.py from
    https://github.com/mvolstad/xethru-python-driver
    https://www.xethru.com/community/resources/xethru-python-driver.33/

    changed it to my own COM port, but I just get 'Starting XeThru sensor... Reset module failed'. When I have XeThru Explorer running in parallel I get 'Starting XeThru sensor... Ready.', but nothing else. I'd expected a "Presence detected." message. It appears the sensor isn't initialized yet? sensor.is_initialized() returning false?
     
  2. Aksel Johnsby

    Aksel Johnsby Moderator Staff Member

    Hi Lovexethru (What a lovely username!)

    XeThru explorer will hold the COM-port and the script won't be able to access it and communicate with the module.

    The GitHub examples you refer to are not maintained for the X4 modules. However you can use the examples found in ModuleConnector.

    Aksel
     
  3. Bjørn Åge Brandal

    Bjørn Åge Brandal Member Staff Member

    Also notice that the X4M300 sensor spend the first 2 minutes building a noise map of the environment (unless noise map feature is turned off). So the module will be in Initialization state during this noise map generation, and only after noise map is completed it will start to output presence/no presence status info.
     
  4. Lovexethru

    Lovexethru New Member

    Thanks Aksel and Bjorn! I tried the shipped ModuleConnector examples for Python on both Windows and Pi 3 and it worked well. I needed to send a message to my own MQTT broker, so wrote a little wrapper around one of your examples. In case it's useful for anyone else:

    Code:
    #!/usr/bin/env python
    
    import subprocess
    import paho.mqtt.client as mqtt
    import paho.mqtt.publish as publish
    import json
    
    ############### MQTT section ##################
    
    Broker = "192.168.1.20"
    pub_topic = "presence"
    
    def on_connect(client, userdata, flags, rc):
        print("Connected MQTT with result code "+str(rc))
    
    client = mqtt.Client()
    client.connect(Broker, 1883, 60)
    client.loop_start()
    
    def publishMQTT(presenceState, distance, signalQuality):
        isPresent = True if presenceState == 'Presence' else False;
    
        jsonObj = json.dumps({'isPresent':isPresent,
                            'presenceState':presenceState,
                                'distance':distance,
                                'signalQuality':signalQuality})
    
        client.publish("presence", str(jsonObj))
    
    ############### Process Sensor section ##################
    
    #filtering for 'Presence -> FrameCounter: 1492, State: Initializing, Distance: 0.0, SignalQuality: 0'
    def processOutput(line):
        stripped = line.strip()       
        print stripped 
        fields = stripped.split()
        isOutputRelevant = len(fields) is 10
        if isOutputRelevant:
            #pic relevant 'words', remove comma
            publishMQTT(fields[5][:-1], fields[7][:-1], fields[9]);
    
    def main():
    
        proc = subprocess.Popen(['python','-u','x4m300_presence_simpleoutput.py', '-d', 'COM6'],stdout=subprocess.PIPE)
        while True:
            line = proc.stdout.readline()
            if line != '':
                processOutput(line)       
            else:
                break
    
    if __name__ == "__main__":
        main()
    BTW: that x4m300_presence_simpleoutput.py sample file contains a 'return x2m200' statement; which AFAIK isn't doing any damage but just FYI.

    Crash after some time

    However, I left this running with the default settings of x4m300_presence_simpleoutput.py, connected to my raspi3 in my living room. There are sensors right next to it saying: 70% humidity/30°C. The program froze after a couple of hours working fine. The chip did feel a bit hotter when touching it; so I cooled it off for a while.

    When I try to start it up again, it worked fine on Windows, but on my raspi3 I now get stuck with this:
    Code:
    1.004184 : Failed to write all bytes to module, expected bytes written: 5, actual bytes written: 0
    Flushing any old data.
    1.012246 : Failed to write all bytes to module, expected bytes written: 6, actual bytes written: 0
    Traceback (most recent call last):
      File "x4m300_presence_simpleoutput.py", line 121, in <module>
        main()
      File "x4m300_presence_simpleoutput.py", line 117, in main
        x4m300_presence_simpleoutput(**vars(options))
      File "x4m300_presence_simpleoutput.py", line 57, in x4m300_presence_simpleoutput
        print("FirmwareID: " + x4m300.get_system_info(XTID_SSIC_FIRMWAREID))
      File "/usr/local/lib/python2.7/dist-packages/pymoduleconnector-1.2.2-py2.7.egg/pymoduleconnector/moduleconnectorwrapper/__init__.py", line 6095, in get_system_info
        return _moduleconnectorwrapper.PyX4M300_get_system_info(self, info_code)
    RuntimeError: Failed to write all bytes to module
    
    Detection Angle/Zone?

    I've put this in a corner of my living room and would like to detect the complete living room in one direction but not up/down (other floor) and behind (other rooms). Is there a way to set this up?
     
  5. Bjørn Åge Brandal

    Bjørn Åge Brandal Member Staff Member

    Thank you for sharing the MQTT code, and letting us know about the 'return x2m200' "bug".

    Are you still experiencing the Crashing? FYI, the X4 SoC integrates power regulation circuitry, so it is normal that it can get a little bit hot under normal operation.

    Wrt detection Angle, the antenna has a fairly wide lobe as shown in the datasheet, and there is no configuration in the sensor to change the detection zone besides adjusting the range. The best positioning to cover a full room would normally be on the wall angling into the room, in one of the corners of the room or in the ceiling in the middle of the room. Basically trying to fit s much of the lobe as possible within the room. Although ceiling mount could mean potentially seeing into the room underneath, the attenuation through the floor into the room below would typically be substantial (depending on the building material), and also there would normally be some distance from the ceiling underneath to any persons walking around. So in practical installations this usually works well.
     
  6. Lovexethru

    Lovexethru New Member

    This worked again; maybe a faulty USB cable?
     
  7. Lovexethru

    Lovexethru New Member

    I'm still getting those '
    Failed to write all bytes to module, expected bytes written: 5, actual bytes written: 0
    Flushing any old data.' errors from time to time. Even if I change cable. Any ideas what this could be and how to workaround/debug this?
     
  8. Bjørn Åge Brandal

    Bjørn Åge Brandal Member Staff Member

    Are you running Windows 7 or Windows 10? We have experienced some USB issues on Windows 7 on the previous version of the X4M300 hardware, that are resolved on windows 10.

    If you PM me the serial number and revison number of your X4M300 (or a picture of your X4M300 module showing all the labels) I can also check if you have one of the affected units, and we can offer you a replacement if the problem continues.
     
  9. Lovexethru

    Lovexethru New Member

    Thanks for this. I've PMed you. Also it worked again.

    And in case it's of use to any other hobbyist out there: attached is a hacky, modified python 3 ModuleConnector 1.4 raspberry pi x4m300_presence_simpleoutput.py, just with some MQTT publishing in it. I've commented all my modifications with '###############'. It also turns the LED off.

    It sends out two MQTT topics - one 'change' boolean only if any presence is either detected or not and one 'detail' JSON that contains that boolean, distance and signalQuality - but it'll only send them if anything has changed compared to the previous frame.

    This is useful for my home - I listen to it on my Node-RED and Home Assistant hubs.

    Code:
    #!/usr/bin/env python
    """ \example x4m300_presence_simpleoutput.py
    This is an example of how to set up and read presence single messages from the
    X4M300 module with the ModuleConnector python wrapper.
    """
    from __future__ import print_function
    import numpy as np
    import time
    from pymoduleconnector import ModuleConnector
    from pymoduleconnector.ids import *
    from time import sleep
    import paho.mqtt.client as mqtt
    import paho.mqtt.publish as publish
    import json
    
    ############### Define MQTT ##################
    
    Broker = "raspberrypi.local"
    pub_change_topic = "presence/human/change/livingroom"
    pub_detail_topic = "presence/human/detail/livingroom"
    
    
    def on_connect(client, userdata, flags, rc):
        print("Connected MQTT with result code " + str(rc))
    
    
    client = mqtt.Client()
    client.connect(Broker, 1883, 60)
    client.loop_start()
    
    
    def publishChange(isPresent):
    
        print("publishChange: " + str(isPresent))
        client.publish(pub_change_topic, str(isPresent).lower(), 1)
    
    
    #@rate_limited(5000)
    def publishDetail(isPresent, presenceState, distance, signalQuality):
    
        print("publishDetail: " + str(presenceState))
        jsonObj = json.dumps({'isPresent': isPresent,
                              'presenceState': presenceState,
                              'distance': distance,
                              'signalQuality': signalQuality})
    
        client.publish(pub_detail_topic, str(jsonObj), 0)
    
    ############### Define MQTT ##################
    
    
    def x4m300_presence_simpleoutput(device_name, detection_zone=(0.5, 9), sensitivity=5, num_messages=0):
    
        # User settings
        detzone_start = detection_zone[0]
        detzone_end = detection_zone[1]
    
        presence_state_text = []
        presence_state_text.append("No presence")
        presence_state_text.append("Presence")
        presence_state_text.append("Initializing")
    
        mc = ModuleConnector(device_name, log_level=0)
        x4m300 = mc.get_x4m300()
    
        sleep(1)  # Allow for MC to read waiting messages from module.
    
        try:
            # Make sure no profile is running.
            x4m300.set_sensor_mode(XTID_SM_STOP, 0)
            print("Stopped already running profile.")
        except RuntimeError:
            # If not initialized, stop returns error. Still OK, just wanted to make sure the profile was not running.
            pass
    
        # Now flush old messages from module
        print("Flushing any old data.")
        while x4m300.peek_message_presence_single():
            presence_single = x4m300.read_message_presence_single()
    
        # Read module info
        print("FirmwareID:", x4m300.get_system_info(XTID_SSIC_FIRMWAREID))
        print("Version:", x4m300.get_system_info(XTID_SSIC_VERSION))
        print("Build:", x4m300.get_system_info(XTID_SSIC_BUILD))
        print("Serial number:", x4m300.get_system_info(XTID_SSIC_SERIALNUMBER))
    
        print("Loading new profile.")
        x4m300.load_profile(XTS_ID_APP_PRESENCE_2)
    
        print("Selecting module output.")
    
        ############### Turn LED off ##################
        x4m300.set_led_control(0, 0)
        ############### Turn LED off ##################
    
        x4m300.set_output_control(XTS_ID_PRESENCE_SINGLE, 1)  # PresenceSingle
        x4m300.set_output_control(
            XTS_ID_PRESENCE_MOVINGLIST, 0)  # PresenceMovingList
    
        print("Setting user settings: DetectionZone = " + str(detzone_start) +
              " to " + str(detzone_end) + ", Sensitivity = " + str(sensitivity))
        x4m300.set_detection_zone(detzone_start, detzone_end)
        x4m300.set_sensitivity(sensitivity)
    
        print("Start profile execution.")
        x4m300.set_sensor_mode(XTID_SM_RUN, 0)  # Make sure no profile is running.
    
        print("Waiting for data...")
    
        ############### Previous State Var ##################
        lastPresenceState = ''
        lastDistance = 0
        lastSignalQuality = 0
    
        presenceState = ''
        distance = 0
        signalQuality = 0
        ############### Previous State Var ##################
    
        n = 0
        while num_messages == 0 or n < num_messages:
            time.sleep(0.1)
    
            while x4m300.peek_message_presence_single():
                presence_single = x4m300.read_message_presence_single()
    
                ############### Call MQTT ##################
    
                presenceState = presence_state_text[presence_single.presence_state]
                distance = str(round(presence_single.distance, 2))
                signalQuality = str(presence_single.signal_quality)
    
                isPresent = True if presenceState == 'Presence' else False
    
                hasPresenceStateChanged = presenceState != lastPresenceState
                if hasPresenceStateChanged:
                    publishChange(isPresent)
    
                if hasPresenceStateChanged or distance != lastDistance or signalQuality != lastSignalQuality:
                    publishDetail(isPresent, presenceState,
                                  distance, signalQuality)
    
                lastPresenceState = presenceState
                lastDistance = distance
                lastSignalQuality = signalQuality
    
                print("Presence ->"
                      + " FrameCounter: " + str(presence_single.frame_counter)
                      + ", State: " + presenceState
                      + ", Distance: " + distance
                      + ", SignalQuality: " + signalQuality
                      )
                ############### Call MQTT ##################
    
                n += 1
    
        x4m300.set_sensor_mode(XTID_SM_STOP, 0)
    
    
    def main():
        import sys
        from optparse import OptionParser
        parser = OptionParser()
        parser.add_option(
            "-d",
            "--device",
            dest="device_name",
            help="device file to use, example: python %s -d COM4" % sys.argv[0],
            metavar="FILE")
    
        parser.add_option(
            "-n",
            "--num-messages",
            dest="num_messages",
            type=int,
            default=0,
            help="how many messages to read (0 = infinite)",
            metavar="INT")
    
        parser.add_option('-z', '--detection_zone', nargs=2, type='float',
                          help='Start and stop of detection zone.', metavar='START STOP',
                          default=(0.5, 9))
    
        parser.add_option('-s', '--sensitivity', nargs=1, type='int',
                          help='Sensor Sensitivity.', metavar='SENSITIVITY',
                          default=5)
    
        (options, args) = parser.parse_args()
    
        if not options.device_name:
            print("Please specify a device name, example: python %s -d COM4" %
                  sys.argv[0])
            sys.exit(1)
        x4m300_presence_simpleoutput(**vars(options))
    
    
    if __name__ == "__main__":
        main()
    
     
    Xióng likes this.
  10. Hi,
    working on getting a RasPi Zero Wi-Fi to run with the XeThru x4m300 and connect it to my Home Assistant setup on another machine...
    Could you share the steps you've done to get your setup up and running? (Do you have a GitHub-repo or have you written about it somewhere else, e.g.?)
    Regards,
    Chr.