update NMEA PAIR response handling

update NMEA PAIR poll handling

update NMEA preset panel

update NMEA preset panel

update config panels grid

Author: semuadmin

README.md

@@ -14,7 +14,7 @@
 [Mapquest API Key](#mapquestapi) |
 [User-defined Presets](#userdefined) |
 [CLI Utilities](#cli) |
-[Known Issues](#knownissues) |
+[Troubleshooting](#troubleshoot) |
 [License](#license) |
 [Author Information](#author)
 
@@ -220,7 +220,7 @@ The UBX Configuration Dialog currently provides the following UBX configuration
 1. Message Rate panel (CFG-MSG) sets message rates per port for UBX and NMEA messages (*legacy protocols only*). Message rate is relative to navigation solution frequency e.g. a message rate of '4' means 'every 4th navigation solution' (higher = less frequent).
 1. Configuration Interface widget (CFG-VALSET, CFG-VALDEL and CFG-VALGET) queries and sets configuration for *modern protocols only*.
 1. UBX Legacy Command configuration panel providing structured updates for a range of legacy CFG-* configuration commands (*legacy protocols only*). Note: 'X' (byte) type attributes can be entered as integers or hexadecimal strings e.g. 522125312 or 0x1f1f0000. Once a command is selected, the configuration is polled and the current values displayed. The user can then amend these values as required and send the updated configuration. Some polls require input arguments (e.g. portID) - these are highlighted and will be set at default values initially (e.g. portID = 0), but can be amended by the user and re-polled using the ![refresh](https://github.com/semuconsulting/PyGPSClient/blob/master/src/pygpsclient/resources/iconmonstr-refresh-lined-24.png?raw=true) button.
-1. Preset Commands widget supports a variety of user-defined UBX commands and queries - see [user defined presets](#userdefined).
+1. Preset Commands widget supports a variety of user-defined UBX commands and queries - see [user-defined presets](#userdefined).
 
 An icon to the right of each 'SEND' 
 ![send icon](https://github.com/semuconsulting/PyGPSClient/blob/master/src/pygpsclient/resources/iconmonstr-arrow-12-24.png?raw=true) button indicates the confirmation status of the configuration command; 
@@ -236,25 +236,26 @@ warning ![warning icon](https://github.com/semuconsulting/PyGPSClient/blob/maste
 **Pre-Requisites:**
 
 - Receiver capable of being configured via proprietary NMEA sentences, connected to the workstation via USB or UART port.
-- The facility includes support for a wide range of Quectel LG and LC series receivers via PQTM*, PSTM* and PAIR* sentences¹. Additional types may be supported in the underlying NMEA parser library [pynmeagps](https://github.com/semuconsulting/pynmeagps) in later releases (*contributions welcome*).
+- The facility includes support for a wide range of Quectel LG and LC series receivers via `PQTM*`, `PSTM*` and `PAIR*` sentences¹ ². Additional types may be supported in the underlying NMEA parser library [pynmeagps](https://github.com/semuconsulting/pynmeagps) in later releases (*contributions welcome*).
 
-¹ Note that Quectel message support depends on the *specific model variant* and firmware version, and several models (e.g. LG290 and LC29) exist in a wide range of variants. Refer to the GNSS Protocol Guide for your *specific* variant.
+   ¹ Note that Quectel receivers implement a bewildering array of different configuration protocols, based on a mixture of proprietary NMEA `PQTM*`, `PSTM*` and `PAIR*` message types. Implementation depends on the *specific model variant and firmware version*, and several models (e.g. LG290 and LC29) exist in a wide range of variants. Refer to the GNSS Protocol Guide for your *specific* variant for details on the available configuration commands.
+
+   ² Note that several Quectel configuration commands require a Hot Restart (PQTMHOT) *or* Save (PQTMSAVEPAR or PAIR513) and Reset (PQTMSRR) before taking effect, including for example PQTMCFGCNST (Enable/Disable Constellations), PQTMCFGFIX (Configure Fix Rate), PQTMCFGSAT (Configure Satellite Masks), PQTMCFGSIGNAL (Configure Signal Masks), PAIR050 (Set Fix Rate) and PAIR864 (Set Baud Rate). Devices (e.g. some LC variants) that don't implement a software reset command may have to be physically disconnected and reconnected.
 
 **Instructions:**
 
 The NMEA Configuration Dialog currently provides the following NMEA configuration panels:
 1. Version panel shows current device hardware/firmware versions (*Double-left-click to refresh*).
 1. Dynamic configuration panel providing structured updates for supported receivers e.g. Quectel LGSERIES via PQTM* sentences, or LCSERIES via PAIR* sentences. Once a command is selected, the configuration is polled and the current values displayed. The user can then amend these values as required and send the updated configuration. Some polls require input arguments (e.g. portid or msgname) - these are highlighted and will be set at default values initially (e.g. portid = 1), but can be amended by the user and re-polled using the ![refresh](https://github.com/semuconsulting/PyGPSClient/blob/master/src/pygpsclient/resources/iconmonstr-refresh-lined-24.png?raw=true) button.
 1. Preset Commands widget supports a variety of user-defined NMEA commands and queries - see [user defined presets](#userdefined).
+1. Preset commands, once selected, can be edited or overwritten in the 'Commands' field before sending, but commands must observe the format `<talker>; <message id>; <payload as comma-separated string>; <msgmode>` (e.g. `P; QTMCFGUART; W,115200; 1` - see [user-defined presets](#userdefined)).
 
 An icon to the right of each 'SEND' 
 ![send icon](https://github.com/semuconsulting/PyGPSClient/blob/master/src/pygpsclient/resources/iconmonstr-arrow-12-24.png?raw=true) button indicates the confirmation status of the configuration command; 
 (pending i.e. awaiting confirmation ![pending icon](https://github.com/semuconsulting/PyGPSClient/blob/master/src/pygpsclient/resources/iconmonstr-time-6-24.png?raw=true), 
 confirmed ![confirmed icon](https://github.com/semuconsulting/PyGPSClient/blob/master/src/pygpsclient/resources/iconmonstr-check-mark-8-24.png?raw=true) or 
 warning ![warning icon](https://github.com/semuconsulting/PyGPSClient/blob/master/src/pygpsclient/resources/iconmonstr-warning-1-24.png?raw=true)).
 
-**NB:** Several Quectel LG and LC series commands require a Hot Restart (PQTMHOT) before taking effect, including PQTMCFGCNST (Enable/Disable Constellations), PQTMCFGFIX (Configure Fix Rate), PQTMCFGSAT (Configure Satellite Masks) and PQTMCFGSIGNAL (Configure Signal Masks). This is a Quectel protocol constraint, not a PyGPSClient issue.
-
 ---
 ## <a name="ttycommands">TTY Configuration Facilities</a>
 
@@ -442,17 +443,21 @@ that this is a User variable rather than a System/Global variable.
 ---
 ## <a name="userdefined">User Defined Presets</a>
 
-The UBX, NMEA and TTY Configuration Dialogs include the facility to send user-defined configuration messages or message sequences to a compatible receiver. These can be set up by adding appropriate comma- or semicolon-delimited message descriptions and payload definitions to the `"ubxpresets_l"`, `"nmeapresets_l"` or `"ttypresets_l"` settings in your json configuration file (see [example provided](https://github.com/semuconsulting/PyGPSClient/blob/master/pygpsclient.json#L189)). The message definition comprises a free-format text description (*avoid embedded commas or semi-colons*) followed by one or more pyubx2 (UBX), pynmeagps (NMEA) or tty (ASCII) message constructors, e,g. 
+The UBX, NMEA and TTY Configuration Dialogs include the facility to send user-defined configuration messages or message sequences to a compatible receiver. These can be set up as follows: 
+
+1. By manually adding appropriate comma- or semicolon-delimited message descriptions and payload definitions to the `"ubxpresets_l"`, `"nmeapresets_l"` or `"ttypresets_l"` sections of your json configuration file (see [example provided](https://github.com/semuconsulting/PyGPSClient/blob/master/pygpsclient.json#L189)). The message definition comprises a free-format text description (*avoid embedded commas or semi-colons*) followed by one or more pyubx2 (UBX), pynmeagps (NMEA) or tty (ASCII) message constructors, e.g. 
 
-- UBX - `<description>, <message class>, <message id>, <payload as hexadecimal string>, <msgmode>`
-- NMEA - `<description>; <talker>; <message id>; <payload as comma-separated string>; <msgmode>`
-- TTY - `<description>; <tty command>`
+   - UBX - `<description>, [<message class>, <message id>, <payload as hexadecimal string>, <msgmode>, ...]`
+   - NMEA - `<description>; [<talker>; <message id>; <payload as comma-separated string>; <msgmode>; ...]`
+   - TTY - `<description>; [<tty command>; ...]`
+
+2. By using the [Configuration Command Load/Save/Record](#recorder) facility to record commands as they are sent to the receiver, and automatically import these recorded commands into the relevant `"...presets_l"` section of the json configuration file.
 
 If the command description contains the term `CONFIRM`, a pop-up confirmation box will appear before the command is actioned.
 
-When PyGPSClient is first started, these settings are pre-populated with an initial set of preset commands, which can be saved to a \*.json configuration file and then manually removed, amended or supplemented in accordance with the user's preferences. To reinstate this initial set at a later date, insert the line `"INIT_PRESETS",` at the top of the relevant `"ubxpresets_l"`, `"nmeapresets_l"` or `"ttypresets_l"` configuration setting.
+When PyGPSClient is first started, the preset command sections are pre-populated in-memory with an initial set of preset commands, which can be saved to a json configuration file and then manually edited in accordance with the user's preferences. To reinstate this initial set at a later date, insert the line `"INIT_PRESETS"` at the top of the relevant `"ubxpresets_l"`, `"nmeapresets_l"` or `"ttypresets_l"` configuration section.
 
-The `pygpsclient.ubx2preset()`, `pygpsclient.nmea2preset()` and `pygpsclient.tty2preset()` helper functions may be used to convert a `UBXMessage`, `NMEAMessage` or ASCII text object into a preset string suitable for copying and pasting into the `"ubxpresents_l":`, `"nmeapresets_l":` or `"ttypresets_l":` JSON configuration sections:
+The `pygpsclient.ubx2preset()`, `pygpsclient.nmea2preset()` and `pygpsclient.tty2preset()` helper functions may be used to convert a `UBXMessage`, `NMEAMessage` or ASCII text object into a string suitable for copying and pasting into the `"ubxpresets_l":`, `"nmeapresets_l":` or `"ttypresets_l":` configuration file sections:
 
 ```python
 from pygpsclient import ubx2preset, nmea2preset, tty2preset
@@ -476,8 +481,6 @@ IM19 System reset CONFIRM; AT+SYSTEM_RESET
 
 Multiple commands can be concatenated on a single line. Illustrative examples are shown in the sample [pygpsclient.json](https://github.com/semuconsulting/PyGPSClient/blob/master/pygpsclient.json#L189) file.
 
-The [Configuration Command Load/Save/Record facility](#configuration-command-loadsaverecord-facility) can also be used to import recorded configuration command sequences into the presets section of the json configuration file.
-
 ---
 ## <a name="cli">Command Line Utilities</a>
 
@@ -486,11 +489,11 @@ The `pygnssutils` and `pyubxutils` libraries which underpin many of the function
 For further details, refer to the `pygnssutils` homepage at [https://github.com/semuconsulting/pygnssutils](https://github.com/semuconsulting/pygnssutils) or `pyubxutils` homepage at [https://github.com/semuconsulting/pyubxutils](https://github.com/semuconsulting/pyubxutils).
 
 ---
-## <a name="knownissues">Known Issues</a>
+## <a name="troubleshoot">Troubleshooting</a>
 
 1. If you encounter persistent `WARNING>>Error parsing data stream Serial stream terminated unexpectedly` messages in the console, this may be indicative of insufficient serial port bandwidth (baudrate or timeout) for the current output message cohort (*particularly if this includes Ephemera or Observation data*). Try increasing the baudrate in the first instance.
 
-2. Most budget USB-UART adapters (e.g. FT232, CH345, CP2102) have a bandwidth limit of around 3MB/s and may not work reliably above 115200 baud, even if the receiver supports higher baud rates. If you're using an adapter and notice significant message corruption, try reducing the baud rate to a maximum 115200.
+2. Most [budget USB-UART adapters](https://www.amazon.co.uk/DSD-TECH-adapter-FT232RL-Compatible/dp/B07BBPX8B8?ref_=ast_sto_dp) (e.g. FT232, CH345, CP2102, *including those embedded on development boards*) have a bandwidth limit of around 3Mbps (≈ 375000 baud) and may not work reliably above 230600 baud, even if the receiver supports higher baud rates. If you're using an adapter and notice significant message corruption (e.g. frequent `WARNING>>..invalid checksum` messages), try reducing the baud rate to a maximum 230600.
 
 3. Some Linux Wayland platforms appear to require Toplevel dialog windows to be non-transient (`transient_dialog_b: 0`) for the window 'maximise' icon to work properly.
 

RELEASE_NOTES.md

@@ -4,6 +4,8 @@
 
 1. Add preset description entry field to Configuration Command Recorder import facility.
 1. Make TTY Presets dialog fully resizeable.
+1. Update NMEA config panel to allow preset commands to be edited or entered manually before sending.
+1. Update NMEA config panel to recognise correct PAIR responses for some Quectel set and poll commands (e.g. a PAIR864 Set baud rate command corresponds to a PAIR865 poll response).
 
 ### RELEASE 1.6.4
 

images/nmeaconfig_widget.png

@@ -98,6 +98,8 @@
 )
 # alternative POLL dictionary names for where POLL command
 # doesn't correspond to SET (fudge for Quectel)
+# e.g. a PAIR864 (set baud rate) command corresponds to PAIR865 (poll baud rate)
+# (for PAIR commands, the poll is typically one digit higher than the set)
 ALT_POLL_NAMES = {
     "AIR050": "AIR051",
     "AIR058": "AIR059",
@@ -111,8 +113,8 @@
     "AIR100": "AIR101",
     "AIR104": "AIR105",
     "AIR400": "AIR401",
-    "AIR410": "AIR421",
-    "AIR420": "AIR411",
+    "AIR410": "AIR411",
+    "AIR420": "AIR421",
     "AIR432": "AIR433",
     "AIR434": "AIR435",
     "AIR436": "AIR437",
@@ -225,7 +227,7 @@ def _body(self):
             command=self._on_refresh,
             font=self.__app.font_md,
         )
-        self._lbl_command = Label(self, text="", width=30, anchor=W)
+        self._lbl_command = Label(self, text="", anchor=W)
         self._frm_container = Frame(self)
         self._can_container = Canvas(self._frm_container)
         self._frm_attrs = Frame(self._can_container)
@@ -246,31 +248,45 @@ def _do_layout(self):
         Layout widgets.
         """
 
-        self._lbl_cfg_dyn.grid(column=0, row=0, columnspan=4, padx=3, sticky=EW)
+        self._lbl_cfg_dyn.grid(column=0, row=0, columnspan=3, padx=3, pady=3, sticky=EW)
         self._lbx_cfg_cmd.grid(
-            column=0, row=1, columnspan=2, rowspan=6, padx=3, pady=3, sticky=EW
+            column=0, row=1, columnspan=1, rowspan=10, padx=3, pady=3, sticky=EW
+        )
+        self._scr_cfg_cmd.grid(column=1, row=1, rowspan=10, sticky=(N, S, W))
+        self._btn_send_command.grid(
+            column=2, row=1, ipadx=3, ipady=3, padx=3, pady=3, sticky=W
+        )
+        self._lbl_send_command.grid(
+            column=2, row=2, ipadx=3, ipady=3, padx=3, pady=3, sticky=W
+        )
+        self._btn_refresh.grid(
+            column=2, row=3, ipadx=3, ipady=3, padx=3, pady=3, sticky=W
+        )
+        self._lbl_command.grid(
+            column=0, row=11, columnspan=3, padx=3, pady=3, sticky=EW
         )
-        self._scr_cfg_cmd.grid(column=1, row=1, rowspan=6, sticky=(N, S, E))
-        self._btn_send_command.grid(column=3, row=1, ipadx=3, ipady=3, sticky=W)
-        self._lbl_send_command.grid(column=3, row=2, ipadx=3, ipady=3, sticky=W)
-        self._btn_refresh.grid(column=3, row=3, ipadx=3, ipady=3, sticky=W)
-        self._lbl_command.grid(column=0, row=7, columnspan=4, padx=3, sticky=EW)
         self._frm_container.grid(
-            column=0, row=8, columnspan=4, rowspan=15, padx=3, sticky=NSEW
+            column=0,
+            row=12,
+            columnspan=3,
+            rowspan=15,
+            padx=3,
+            pady=3,
+            sticky=NSEW,
         )
         self._can_container.grid(
-            column=0, row=0, columnspan=3, rowspan=15, padx=3, sticky=NSEW
+            column=0,
+            row=0,
+            columnspan=3,
+            rowspan=15,
+            padx=3,
+            pady=3,
+            sticky=NSEW,
         )
-        self._scr_container_ver.grid(column=3, row=0, rowspan=15, sticky=(N, S, E))
+        self._scr_container_ver.grid(column=3, row=0, rowspan=15, sticky=(N, S, W))
         self._scr_container_hor.grid(
-            column=0, row=15, columnspan=4, rowspan=15, sticky=EW
+            column=0, row=15, columnspan=3, rowspan=15, sticky=EW
         )
-
-        cols, rows = self.grid_size()
-        for i in range(cols):
-            self.grid_columnconfigure(i, weight=1)
-        for i in range(rows):
-            self.grid_rowconfigure(i, weight=1)
         self.option_add("*Font", self.__app.font_sm)
 
     def _attach_events(self):
@@ -406,7 +422,7 @@ def _do_poll_cfg(self, *args, **kwargs):  # pylint: disable=unused-argument
             return
 
         msg = penddlg = pendcfg = None
-        # use alternate names for some NMEA PAIR/PQTM POLL commands
+        # use alternate name for some NMEA PAIR/PQTM POLL commands
         cfg_id = ALT_POLL_NAMES.get(self._cfg_id, self._cfg_id)
         # set any POLL arguments to specified or default values
         # e.g. portid = "1", msgname="RMC"
@@ -477,23 +493,26 @@ def _do_poll_args(self, cfg_id: str) -> dict:
             pass
         return args
 
-    def update_status(self, msg: object):
+    def update_status(self, msg: NMEAMessage | UBXMessage):
         """
         UBXHandler or NMEAHandler module has received expected command response
         and forwarded it to this module, entry widgets are pre-populated with
         current configuration values and confirmation status is updated.
 
-        :param object msg: UBXMessage or NMEAMessage response
+        :param NMEAMessage | UBXMessage msg: UBXMessage or NMEAMessage response
         """
 
+        # self.logger.debug(f"{msg.identity=}")
         ok = False
-        # strip off any variant suffix from cfg_id
-        # e.g. "QTMCFGUART_CURR" -> "PQTMCFGUART"
-        cfg_id = (
-            "P" + self._cfg_id.rsplit("_", 1)[0]
-            if self._protocol == NMEA
-            else self._cfg_id
-        )
+        if self._protocol == NMEA:
+            # use alternate name for some NMEA PAIR/PQTM POLL commands
+            # e.g. a PAIR864 (set baud rate) command corresponds to PAIR865 (poll baud rate)
+            cfg_id = ALT_POLL_NAMES.get(self._cfg_id, self._cfg_id)
+            # strip off any variant suffix from cfg_id
+            # e.g. "QTMCFGUART_CURR" -> "PQTMCFGUART"
+            cfg_id = "P" + cfg_id.rsplit("_", 1)[0]
+        else:
+            cfg_id = self._cfg_id
 
         # if this message identity matches an expected response
         if msg.identity in (cfg_id, ACK, NAK):
@@ -530,13 +549,13 @@ def _clear_widgets(self):
             wdg.destroy()
             wdg = None
         Label(self._frm_attrs, text="Attribute", width=12, anchor=W).grid(
-            column=0, row=0, padx=3, sticky=(W)
+            column=0, row=0, padx=3, sticky=W
         )
         Label(self._frm_attrs, text="Value", width=20, anchor=W).grid(
-            column=1, row=0, padx=3, sticky=(W)
+            column=1, row=0, padx=3, sticky=W
         )
         Label(self._frm_attrs, text="Type", width=5, anchor=W).grid(
-            column=2, row=0, padx=3, sticky=(W)
+            column=2, row=0, padx=3, sticky=W
         )
 
     def _add_widgets(self, pdict: dict, row: int, index: int) -> int:

src/pygpsclient/dynamic_config_frame.py

@@ -271,6 +271,8 @@
 VALDMY = 8
 VALLEN = 9
 VALBOOL = 10
+VALREGEX = 11
+VALCUSTOM = 12
 WAYPOINT = "waypoint"
 WIDGETU1 = (200, 200)  # small widget size
 WIDGETU2 = (300, 200)  # medium widget size