Developer Notes from Before January 27, 2021¶
Note
These notes have been edited to remove some of no longer useful information. In some cases, the original notes have been incorporated into the application. In other cases, the notes reflected the understanding at the time which turned out to be wrong
Plan For Starting the Application:
Note: To control relays, need to be able to select them by topology it turns out. Almost nothing that I’m interested in has a persistent unique identifier like a serial number attached to it.
✔ Create USB utilities to understand how to USB and PCI work.
✔ Create a print_tree routine to visualize topology
✔ Factor out utilities to make it feasible to uniquely identify relays.
✔ Create some REST API to drive adoption of Rasberry Pi into ecosystem.
✔ This requires a persistence layer, so do SQLAlchemy next
✔ Transfer python package to RaspPi.
Web inteface to control lights
Pick out colors of lights with sensor
✔ Use Sphinx to generate styling for dynamic pages.
Regarding Using the nmap Utility¶
To get more information from nmap, it should be run using sudo.
Found this searching the web:
While OP might have a good reason for wanting to do exactly this, it usually is a bad idea (password can be read by ps, and so on) and I wanted to provide a more secure alternative.
A better solution if you want to run something with sudo without putting in your password is to allow your user to do exactly that one command without password.
Open sudoers file with sudo visudo and add the following line (obviously replace the username at the beginning and the command at the end):
alice ALL = NOPASSWD: /full/path/to/command
This is explained more here: https://askubuntu.com/a/39294
TODO:
Capture modified version of sudoers
Document changes needed to get it to work without a password.
Detect and prompt user to install it if it is not available.
Pytest¶
pytest -s -p no:logging
pytest -vv -s –log-cli-level=DEBUG –log-cli-format=”%(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)” –log-cli-date-format=”%Y-%m-%d %H:%M:%S”
Fix it up so path is full.
Olio¶
Linter rstcheck is not installed.
Serial Communication with USB Relay¶
(venv) rich@wendy:~/src/github.com/rich-dobbs-13440/astutus/pytests$ lsusb -d 1a86:7523
Bus 010 Device 022: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter
Bus 010 Device 021: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter
(venv) rich@wendy:~/src/github.com/rich-dobbs-13440/astutus/pytests$ sudo python3
>>> port = serial.Serial("/dev/ttyUSB1")
>>> port.baudrate = 9600
>>> port.bytesize = 8
>>> port.parity = 'N'
>>> port.stopbits = 1
>>> off = bytearray(b'\xA0\x01\x01\xA2')
>>> on = bytearray(b'\xA0\x01\x00\xA1')
>>> num = port.write(off)
>>> num = port.write(on)
>>> num = port.write(off)
>>> num = port.write(on)
>>> port.close()
>>> exit()
(venv) rich@wendy:~/src/github.com/rich-dobbs-13440/astutus/pytests$ ls -l /dev/ttyUSB0
crw-rw---- 1 root dialout 188, 0 Dec 23 14:21 /dev/ttyUSB0
(venv) rich@wendy:~/src/github.com/rich-dobbs-13440/astutus/pytests$ ls -l /sys/dev/char/188:0
lrwxrwxrwx 1 root root 0 Dec 23 15:15 /sys/dev/char/188:0 -> ../../devices/pci0000:00/0000:00:07.0/0000:05:00.0/usb10/10-1/10-1.2/10-1.2.3/10-1.2.3:1.0/ttyUSB0/tty/ttyUSB0
(venv) rich@wendy:~/src/github.com/rich-dobbs-13440/astutus/pytests$ ls -l /dev/ttyUSB0
crw-rw---- 1 root dialout 188, 0 Dec 23 14:21 /dev/ttyUSB0
(venv) rich@wendy:~/src/github.com/rich-dobbs-13440/astutus/pytests$ ls -l /sys/dev/char/188:0
lrwxrwxrwx 1 root root 0 Dec 23 15:15 /sys/dev/char/188:0 -> ../../devices/pci0000:00/0000:00:07.0/0000:05:00.0/usb10/10-1/10-1.2/10-1.2.3/10-1.2.3:1.0/ttyUSB0/tty/ttyUSB0
(venv) rich@wendy:~/src/github.com/rich-dobbs-13440/astutus/pytests$ ls ../../devices/pci0000:00/0000:00:07.0/0000:05:00.0/usb10/10-1/10-1.2/10-1.2.3/10-1.2.3:1.0/devnum
ls: cannot access '../../devices/pci0000:00/0000:00:07.0/0000:05:00.0/usb10/10-1/10-1.2/10-1.2.3/10-1.2.3:1.0/devnum': No such file or directory
(venv) rich@wendy:~/src/github.com/rich-dobbs-13440/astutus/pytests$ cd /sys/devices/pci0000:00(venv) rich@wendy:/sys/devices/pci0000:00$ ls
0000:00:00.0 0000:00:05.0 0000:00:11.0 0000:00:13.0 0000:00:14.2 0000:00:14.5 0000:00:18.0 0000:00:18.3 firmware_node power
0000:00:02.0 0000:00:06.0 0000:00:12.0 0000:00:13.2 0000:00:14.3 0000:00:16.0 0000:00:18.1 0000:00:18.4 pci_bus uevent
0000:00:04.0 0000:00:07.0 0000:00:12.2 0000:00:14.0 0000:00:14.4 0000:00:16.2 0000:00:18.2 0000:00:18.5 PNP0C14:00
(venv) rich@wendy:/sys/devices/pci0000:00$ cd 0000:00:07.0
(venv) rich@wendy:/sys/devices/pci0000:00/0000:00:07.0$ ls
0000:05:00.0 consistent_dma_mask_bits device enable local_cpus msi_bus remove revision subsystem_device
broken_parity_status current_link_speed dma_mask_bits firmware_node max_link_speed numa_node rescan secondary_bus_number subsystem_vendor
class current_link_width driver irq max_link_width pci_bus reset subordinate_bus_number uevent
config d3cold_allowed driver_override local_cpulist modalias power resource subsystem vendor
(venv) rich@wendy:/sys/devices/pci0000:00/0000:00:07.0$ cd 0000\:05\:00.0/
(venv) rich@wendy:/sys/devices/pci0000:00/0000:00:07.0/0000:05:00.0$ ls
broken_parity_status current_link_speed dma_mask_bits irq max_link_width numa_node rescan revision uevent
class current_link_width driver local_cpulist modalias pools reset subsystem usb10
config d3cold_allowed driver_override local_cpus msi_bus power resource subsystem_device usb11
consistent_dma_mask_bits device enable max_link_speed msi_irqs remove resource0 subsystem_vendor vendor
(venv) rich@wendy:/sys/devices/pci0000:00/0000:00:07.0/0000:05:00.0$ cd usb10
(venv) rich@wendy:/sys/devices/pci0000:00/0000:00:07.0/0000:05:00.0/usb10$ ls
10-0:1.0 bcdDevice bmAttributes busnum devpath interface_authorized_default product speed
10-1 bConfigurationValue bMaxPacketSize0 configuration driver ltm_capable quirks subsystem
authorized bDeviceClass bMaxPower descriptors ep_00 manufacturer removable uevent
authorized_default bDeviceProtocol bNumConfigurations dev idProduct maxchild remove urbnum
avoid_reset_quirk bDeviceSubClass bNumInterfaces devnum idVendor power serial version
(venv) rich@wendy:/sys/devices/pci0000:00/0000:00:07.0/0000:05:00.0/usb10$ cd busnum
bash: cd: busnum: Not a directory
(venv) rich@wendy:/sys/devices/pci0000:00/0000:00:07.0/0000:05:00.0/usb10$ cat busnum
10
(venv) rich@wendy:/sys/devices/pci0000:00/0000:00:07.0/0000:05:00.0/usb10$ cat devnum
1
(venv) rich@wendy:/sys/devices/pci0000:00/0000:00:07.0/0000:05:00.0/usb10$
(venv) rich@wendy:/sys/devices$ grep -r . -e "1a86" 2>/dev/null
Bus (\d+) Device (\d+)
Bus (\d+) Device (\d+): ID ([0-9,af]{4}):([0-9,a-f]{4}) (.*)
https://askubuntu.com/questions/373096/how-do-i-permanently-change-permissions-for-dev-ttys0#373269
There’s no need to change system file’s permissions. The serial devices have the following default permissions:
crw-rw—- 1 root dialout … /dev/ttyS0 So all you have to do is add the user to the dialout group:
sudo adduser $USER dialout
rich@wendy:~$ lsusb --tree
/: Bus 11.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 5000M
/: Bus 10.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 1: Dev 3, If 0, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 1: Dev 3, If 1, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 2: Dev 4, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 1: Dev 11, If 0, Class=Vendor Specific Class, Driver=ch341, 12M
|__ Port 4: Dev 9, If 0, Class=Vendor Specific Class, Driver=r8152, 480M
|__ Port 2: Dev 13, If 2, Class=Audio, Driver=snd-usb-audio, 12M
|__ Port 2: Dev 13, If 0, Class=Audio, Driver=snd-usb-audio, 12M
|__ Port 2: Dev 13, If 3, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 2: Dev 13, If 1, Class=Audio, Driver=snd-usb-audio, 12M
|__ Port 3: Dev 15, If 0, Class=Imaging, Driver=usbfs, 480M
|__ Port 3: Dev 14, If 0, Class=Vendor Specific Class, Driver=ch341, 12M
|__ Port 4: Dev 7, If 0, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 4: Dev 7, If 3, Class=Video, Driver=uvcvideo, 480M
|__ Port 4: Dev 7, If 1, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 4: Dev 7, If 2, Class=Video, Driver=uvcvideo, 480M
/: Bus 09.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 5000M
/: Bus 08.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
|__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 1: Dev 2, If 1, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 1: Dev 2, If 2, Class=Human Interface Device, Driver=usbhid, 12M
/: Bus 07.Port 1: Dev 1, Class=root_hub, Driver=ohci-pci/4p, 12M
/: Bus 06.Port 1: Dev 1, Class=root_hub, Driver=ohci-pci/2p, 12M
/: Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ohci-pci/5p, 12M
/: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=ohci-pci/5p, 12M
/: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/4p, 480M
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/5p, 480M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/5p, 480M
rich@wendy:~$ lsusb
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 011 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 010 Device 007: ID 046d:082c Logitech, Inc. HD Webcam C615
Bus 010 Device 014: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter
Bus 010 Device 020: ID 0bda:8153 Realtek Semiconductor Corp. RTL8153 Gigabit Ethernet Adapter
Bus 010 Device 030: ID 14cd:125d Super Top
Bus 010 Device 022: ID 0e6f:0232 Logic3
Bus 010 Device 031: ID 04e8:6860 Samsung Electronics Co., Ltd Galaxy series, misc. (MTP mode)
Bus 010 Device 016: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 010 Device 003: ID 046d:c52f Logitech, Inc. Unifying Receiver
Bus 010 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 010 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 009 Device 003: ID 0bda:8153 Realtek Semiconductor Corp. RTL8153 Gigabit Ethernet Adapter
Bus 009 Device 002: ID 05e3:0612 Genesys Logic, Inc. Hub
Bus 009 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 008 Device 007: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 008 Device 006: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 008 Device 003: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 008 Device 002: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 008 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/try-astutus/astutus/packaging/dist/venv/lib/python3.8/site-packages/astutus/web/static/_docs/source/developer_notes.html'
Deprecation Warning¶
2020-12-28 18:36 Warning showed up in Pytest with most recent venv:
================================================================================================ warnings summary =================================================================================================
../venv/lib/python3.8/site-packages/future/standard_library/__init__.py:65
/home/rich/src/github.com/rich-dobbs-13440/astutus/venv/lib/python3.8/site-packages/future/standard_library/__init__.py:65: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
-- Docs: https://docs.pytest.org/en/stable/warnings.html
Different Selector Syntax Idea¶
pci(0x1002:0x5a1b)/pci(0x1b21:0x1042)/usb(1d6b:0002)/usb(05e3:0610)/usb(1a86:7523)
Convert node to list
Do a match based on that.
Add a capability to astutus-usb-tree to list out paths for a selected node.
Do it in JSON format, that can be plopped into aliases and then modified.
Favicon For Website¶
First, use the download button to download the files listed below. Place the files in the root directory of your website:
android-chrome-192x192.png
android-chrome-512x512.png
apple-touch-icon.png
favicon-16x16.png
favicon-32x32.png
favicon.ico
site.webmanifest
Next, copy the following link tags and paste them into the head of your HTML.
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png">
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png">
<link rel="manifest" href="/site.webmanifest">
Installing On Raspberry Pi¶
Could not find a version that satisfies the requirement treelib (from astutus) (from versions: )
No matching distribution found for treelib (from astutus)
Collecting SQLAlchemy>=0.8.0 (from flask-sqlalchemy->astutus)
Could not find a version that satisfies the requirement SQLAlchemy>=0.8.0 (from flask-sqlalchemy->astutus) (from versions: )
No matching distribution found for SQLAlchemy>=0.8.0 (from flask-sqlalchemy->astutus)
Collecting pyyaml>=3.13 (from serial->astutus)
Could not find a version that satisfies the requirement pyyaml>=3.13 (from serial->astutus) (from versions: )
No matching distribution found for pyyaml>=3.13 (from serial->astutus)
pip3 install --no-index --find-links=. astutus
Dynamic TOC Tree¶
Here is structure of toc generated on the page for the toctree directive:
<div class="toctree-wrapper compound">
<p class="caption"><span class="caption-text">Contents:</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="flask_app_dyn_usb_device.html">USB Device Tree</a></li>
<li class="toctree-l1"><a class="reference internal" href="flask_app_dyn_usb_alias.html">USB Aliases</a></li>
<li class="toctree-l1"><a class="reference internal" href="flask_app_dyn_usb_configuration.html">Device Configurations</a></li>
</ul>
</div>