Mecrisp-Stellaris Forth IDE for the Feather RP2040

,

Where I describe how I develop Forth on my Mac using Sublime Text 3 and Serial, as my Forth IDE.

Summary

  1. Edit Forth code in Sublime Text 3 using Textmate Forth bundle for code highlighting.
  2. Execute ST3’s Build command which transfers the code to the board for on-board compiling for Forth, typically a 2-3 second transfer (1500 bytes/sec).
  3. Use the macOS app, Serial for communicating with Forth on the board.

Sources

On the Shoulders of Giants

In Terry’s comments as to a Forth IDE, he goes into great detail as to the speed of transmission, the debugging capabilities and register documentation of his prescribed IDE. I honor what he describes and it certainly is a powerful and capable IDE. That said, I’ve been searching for a way to do development using my system (Mac vs. Linux), my tools (Sublime Text vs. Vim) and my board (RP2040-based vs. STM).

The sources above were all great inspirations to my end product. I recommend you take a look at them to gain a better understanding of what is capable. In the meantime, I give my version of my Forth IDE (for now):

Editing: Sublime Text 3 (ST3)

I edit my file in ST3, using an old TextMate bundle for Forth. It does a pretty good job of highlighting Forth syntax properly. Once I’ve finished editing, I need to send the text file to the Feather RP2040.

ST3 has the ability to automate a build, whether its for a C++ program or Python. Or in my case, “build” is simply transferring the file to the board and having Forth compile it. I wrote a absurdly simple build program and saved it per ST3’s requirements, I saved it in

"~/Library/Application Support/Sublime Text 3/Packages/User/forth.sublime-build":

{
    "cmd": ["/Users/lkoepsel/Documents/python/forth/upload.py", "$file"],
    "selector": "source.forth",
}

Forth Transfer Examples

I reviewed upload.c, xfr.py and ff_shell.py, then created my own (albeit Frankenstein version) of an upload program. Some notes as to what I liked about each:

upload.c

Jan does a nice job of documenting the patches required to the Mecrisp-Stellaris code. He makes specific changes to the interpreter and datastackandmacro code so that it can be more efficient with the uploader. I haven’t incorporated all of his changes, I use a different acknowledgement scheme for now. He uses ACK/NAK which is a great way to do it and I’m going to look at for my code in the future.

xfr.py

While attempting to fix some things in upload.c, I ran across this note by Piotr regarding an absurdly simple program xfr.py:

#!/usr/bin/env python3
import sys
for l in open(sys.argv[1], "rt"):
    sys.stdout.write(l)
    r = sys.stdin.readline()
    sys.stderr.write(r)
    if not r.endswith("ok.\n"):
        sys.exit(1)

You can invoke this using picocom:

picocom -b 115200 /dev/ttyACM0 --imap lfcrlf,crcrlf --omap delbs,crlf --send-cmd "./xfr.py"

And it works, quite well! This source on Github is his more fleshed out version. It works extremely well and was my solution for a period of time.

However, once I finally understood how to increase the RP2040 baud rate, I wasn’t able to run picocom at 921600 on my Mac. Which meant, I needed a more full-feature version, one that didn’t require a terminal program.

ff_shell.py

This version works well with a previous version of Forth, called FlashForth. I really like FF, however, Mecrisp-Stellaris Forth is significantly more advanced and supported so I’ve made the move.

This program uses PySerial to communicate directly with the board and PySerial was the missing link. Once I began to use PySerial as the basis for communication, it just seemed like the way to go.

upload.py

All of this resulted in upload.py, which is a stand-alone app which is the basis of the build function in ST3.

#!/usr/bin/env python3
import argparse
import serial
import sys
import os
import re
import datetime
import traceback


class Config(object):
    def __init__(self):
        self.serial_port = '/dev/cu.usbserial-A10K59P8'
        self.rate = '921600'
        self.hw = False
        self.sw = False
        self.chardelay = '0'
        self.newlinedelay = '0'
        self.charflowcontrol = False


def serial_open(config):
    if config.cfg:
        print("Port:"
              + str(config.port)
              + " Speed:"
              + str(config.rate)
              + " hw:"
              + str(config.hw)
              + " sw:"
              + str(config.sw)
              + " newlinedelay:"
              + str(config.chardelay)
              + " chardelay:"
              + str(config.chardelay)
              + " charfc:"
              + str(config.charflowcontrol))
    try:
        config.ser = serial.Serial(config.port, config.rate, timeout=1,
                                   rtscts=config.hw,
                                   xonxoff=config.sw)
    except serial.SerialException as e:
        print('Could not open serial port', config.serial_port, 'due to:',
              os.strerror(e.errno))
        # raise e


def parse_arg(config):
    parser = argparse.ArgumentParser(
        description="Upload app for Mecrisp-Stellaris Forth",
        epilog="""Upload to board at higher speeds.""")
    parser.add_argument("file", metavar="FILE", help="file to send")
    parser.add_argument("--config", "-f", action="store_true",
                        default=False,
                        help="Print port configuration")
    parser.add_argument("--port", "-p", action="store",
                        type=str, default='/dev/cu.usbserial-A10K59P8',
                        help="Serial port name")
    parser.add_argument("--hw", action="store_true",
                        default=False, help="Serial port RTS/CTS enable")
    parser.add_argument("--sw", action="store_true",
                        default=False, help="Serial port XON/XOFF enable")
    parser.add_argument("--speed", "-s", action="store",
                        type=str, default=921600, help="Serial port speed")
    parser.add_argument("--chardelay", "-d", action="store",
                        type=str, default=0,
                        help="Character delay(milliseconds)")
    parser.add_argument("--newlinedelay", "-n", action="store",
                        type=str, default=0,
                        help="Newline delay(milliseconds)")
    parser.add_argument("--cc", "-c", action="store_true",
                        default=False,
                        help="Character by character flow control")
    parser.add_argument("-t", "--print-statistics", action="store_true",
                        default=True, help="print transfer statistics")
    parser.add_argument("-I", "--include-path", metavar="DIR", action="append",
                        default=["."],
                        help="append directory to include paths")

    arg = parser.parse_args()
    config.cfg = arg.config
    config.file = arg.file
    config.port = arg.port
    config.hw = arg.hw
    config.sw = arg.sw
    config.rate = arg.speed
    config.chardelay = arg.chardelay
    config.newlinedelay = arg.newlinedelay
    config.charflowcontrol = arg.cc
    config.stats = arg.print_statistics
    config.incl = arg.include_path


# specific function to show hex ASCII values for hidden characters on error
def rtoASCII(r):
    print(str(bytearray(r)))
    return " ".join(str(hex(char)) for char in r)


def main():

    config = Config()
    parse_arg(config)
    serial_open(config)
    t0 = datetime.datetime.now()
    n = xfr(None, 0, config)
    elapsed_time = datetime.datetime.now() - t0
    speed = int(n[0] / elapsed_time.total_seconds())
    if config.stats:
        print(f'*** lines read: {n[1]}')
        print(f"*** elapsed time: {elapsed_time} ({speed} bytes/s) ***")


def xfr(parent_fname, parent_lineno, config):

    comments = re.compile(r'^\s*\\ .*')
    empty = re.compile(r'^\s*$')
    lineno = 1
    prevlines = 1
    prev = 8
    n_bytes_sent = 0
    response = []
    orig_file = ['Start']
    try:
        for line in open(config.file, "rt"):
            if lineno > prev:
                prevlines = lineno - prev
            else:
                prevlines = lineno - lineno
            orig_file.append(line)
            if (not comments.match(line)) and (not empty.match(line)):
                config.ser.write(str.encode(line))
                n_bytes_sent += len(line)
                response_line = config.ser.readline()
                response.append(response_line)
                if not response_line.endswith(b'ok.\r\n'):
                    print(f'*** Compilation error, line: {lineno}')
                    sys.stderr.write(rtoASCII(response_line))
                    for line in range(prevlines, lineno + 1):
                        print(line, orig_file[line], end='')
                    sys.exit()
            lineno += 1
        return [n_bytes_sent, lineno]
    except (OSError) as e:
        if parent_fname is not None:
            sys.stderr.write(f"*** {parent_fname}({parent_lineno}): {e} ***")
        else:
            sys.stderr.write(f"*** : {e} ***")
        sys.exit(1)


try:
    sys.exit(main())
except Exception as e:
    traceback.print_exc()
    print("sys.exit {0}".format(e))

Comments powered by Talkyard.