Here is a collection of my thought and findings from the last few years. I write my software for the Spectrum using Z88DK in C leveraging libraries to support a range of Spectrum setups. I can not claim credit for the techniques, I learned from Thomas Cherryhomes with his IRATA.online project. ESP WiFi modems are appearing for retro computers with offers to connect your old system to the internet. These are not the only option, and some Bus connected devices can connect to the "internet". So cutting through the noise are there some common abstractions that can define "standards" or at least common terminology? All these devices are aiming to provide access to TCP/IP networks, thus enabling internet access for software. Our first point needs to be these adapters are providing SOFTWARE access to the internet. Unless the device has some built-in ROM or Software delivering an application, additional software is required. So you need to know what software is available or compatible with the hardware device. Networking is dependent on both hardware and software; you need both. The hardware provides an API for software applications to use. In terms of modern networking, this would be the Socket API. BUS based network interfaces might adopt a socket API, for example, Spectranet. These devices are what could be considered more like a network card. There is a newer approach to this API using AT Commands, which has come about with retro computing a hardware approach to bridge old modem-based software over the internet in place of phonelines. The rising confusion with the "WiFi network adapter", is they obfuscate the Serial port connection and AT Commands. So lets clear the clutter and identify what these devices are. Initially, the intent was to connect to BBS systems using old Serial Terminal software. However, the approach can connect to any remote TCP/IP socket. Therefore it is possible for other uses beyond BBS systems and terminals, assuming suitable software. "Serial Internet AT (WiFi) Modem", connects to the serial port (of a computer). They realise an API based on the AT Hayes modem command set. Software applications communicate over a serial port with the device, sending AT commands to make TCP/IP socket connections. Offloading the TCP/IP stack to the "modem"; thus, the software only needs to implement the application protocol (FTP/Gopher/HTTP). "Network adapter", connects to the system BUS/IO subsystem and implements a Socket based API. Assuming here the TCP/IP stack has been offloaded to the Network adapter (Wiznet chip for example). The application only needs to implement the protocol required. Given most retro systems have only a few Kilobytes of memory, running a full TCP/IP stack on the target computer is less than desirable! What about a BUS connected device that implements AT Commands? Entirely possible, someone that likes the AT command approach to network programming. If the hardware does not use a UART or similar serial interface between the BUS and TCP/IP stack, then it would be a true hybrid. Else, if there is some form of Serial hardware, then from a software standpoint the solution is just another Serial port device and AT Modem. It is essential software developers understand the two paradigms at play here. If you plan on writing a network application, you can design the software in such a way as to make it possible to support them all. Here is a short list of points to consider when network programming or hardware building. Abstract the Transmit and Recieve routines so they can work with Socket or AT command APIs. Does the target system have more than one type of serial port? Are you support one or all with your AT Command based application. If the serial port is software-defined, then handshaking might be required for the hardware. Often serial connections with microcontroller and alike use three-wire solutions; this is not always true! Network protocols handle connections differently regard to the connection being continuous or open/closed. Telnet sessions are an example of streaming; a link is opened and held open. HTTP opens a connection data is transfered, the connection closes. Being aware of this is vital for two reasons. AT Command software will need to frequently "re-dial" connections to fetch data. Socket API software will need to empty the receive buffer when reading data. Partially reading a receive buffer when a socket closes can lose data. Data loss like this could apply to some AT Command approaches too. Encryption or lack of is a big thing to remember. To date, there are no solutions for connecting to encrypted services. HTTPS access from a retro computer is not possible. Similarly, we need to remember any applications built are vulnerable to being sniffed. Serial Internet AT Modem solutions has the advantage here in being an architecture that could quickly implement SSL. ESP chips are not the only way to create these devices.
0 Comments
This year has been a roller coaster, and that is putting it mildly. Attended my first conference this year, presenting in the Doctoral student symposium. I was probably more nervous about leaving for the conference than actually presenting. The feedback was excellent, mainly points and questions that I had answers too. "I seem to be well-read" was the closing comment, which I take as good sign! Other than that, I have met the worlds most exceptional minds in the Modelling community, and they seem like a friendly bunch. I managed to get a Spectranet interface from Ben over at ByteDelight. This might have had me distracted for a few hours and kick-started another little software project. It is well worth the wait, and Dylan sure as hell was ahead of the curve when he designed it all those years ago! ZXGo4it is my latest little project, a gopher client for the spectrum. So far it is a working prototype not released, there is a demo kicking about in the Spectranet group on Facebook. I should probably look to get the videos I make on here somehow, maybe off a YouTube channel? Back to the topic, so the gopher protocol is a bit of a dead protocol these days, but once upon a time, it was the other protocol rivalling HTTP and HTML. The specification for gopher is actually straightforward, based on formatted lines to form pages as menu indexes. Each line is recommended to be 74 characters long to enable access from 80 column terminal. Yes, gopher is intended as text only 80 columns which is almost practical for a speccy! The downside is the specification doesn't limit page sizes, the largest I have seen is 50K (bigger than a speccy 48K!). This project turned into an excellent refresher on Pointer, Malloc and Structs in C. The dynamic memory allocation and linked list for handling the page data are working really well. When a page contains more lines than the 16K heap lines are discarded without crashing the program! Next big challenge is to rework the prototype so it can handle pages larger than the heap. I have a few bright ideas that might work, not giving away any spoilers on the magic yet ;) I have not been doing an outstanding job at posting on this blog! (TL:DR Pictures at the bottom)
Work has been resumed on SnapCterm. I seem to be getting into a flow with it which is good. Rolling through some less dramatic changes, iterating over sections and re-writing bits. Moving out bits to functions and general housekeeping that is hardly exciting reading. I gave in to actually processing the ESC codes and implementing what might be a state machine? Point of technicality, basically we start flagging and filtering through IFs as each character passes from RX buffer to screen. That said I haven't fully detached the RX Buffer from the screen, poping data from the buffer still gets posted to ANSI screen driver. Only now there is a process to watch the stream. So we can push other ESC codes to the screen to correct or respond to the remote system (which wasn't implemented in the ANSI driver!). Not quite at a point of calling this section of code done, I may go for a full disconnect, process the ESC code and modify before posting to screen. Now for something a little more visual, colour clash correction! Oh, the problem with a single Bright bit for both foreground and background. So ANSI terminal bright seems to ONLY apply to the foreground colour. So blue on blue with "Bold" enabled gives a light blue text on a blue background. On the spectrum colour pallet that becomes a beautiful bright blue text on a nice bright blue background... Black on black is also very popular, doubly bugged on a spectrum. Not only does the bright apply foreground and background colour, but "bright black" is also the same as black... Took a while to figure out a process to handle this problem. Comes down to fully decoding the ESC code for setting colour and attributes. Keeping track of them in a set of variables. When we have a Bold enabled check the foreground colour against the background. If they match look up the colour code and push in an ESC code to change the colours. On the next ESC code that changes the colours, recall what the setting should be and replay them. Then because I pass characters direct to the ANSI driver, we then need to replay the last code sequence again! As the first processing updated the changed set, which might not have replaced all colours and attribute. This is one reason why I might need to re-think the design. Any way picture time... Spot the original ANSI render using Mate Terminal in Linux. SnapCterm without correction and with correction on the Spectrum. Good bye 2018... So job wrapped up, online enrolment is done 2nd January I walk into a new building and all new "job". It has been a busy period as December usually is, the Christmas bit seeing friends and Family. Plato development has been back seat for this month, as have a lot of things. I managed to squeeze in some time on SnapCterm which has built some knowledge for Plato. Nothing groundbreaking but a few bits on timing and the effects of buffering on serial communications. Screen drawing is extremely slow! ANSI and the ASCII set is not quite so alien anymore. I have managed to straighten out SnapCterm to respond correctly to some ESC codes, implemented cursor keys, a buffer for serial comms and fixed that attribute issue. It isn't quite ready for release still need to implement some methods for CTRL ALT ESC keys and combinations of. FUSE and the serial port are now working (IF1 only) using CURL, while its sad to not test on real hardware. It has helped to leap through some work quickly in the short time available. Roll on 2019 I am ready to crack on with it. So life got a little busy. This month I have resigned from my job and accepted an offer for a studentship (PhD). Yes its caused some paperwork and kicked the day job up into high gear. On top of the usual children's birthdays and find a school for the first born.
So general short update. PLATOterm on the ZX is looking good and a few bugs removed. Thom is thinking about a large overhaul. I have started looking at buffering the RX data, which after some testing does seem to improve performance. The size of the buffer and time to fill needs to fit with the screen timings. Generally leads me to think there is a need to time to code more efficiently to the screen. There could be some performance gains from the screen draw routines, I have yet to venture into that code. I opened up a small project called snapCterm, which is a ANSI 80 column terminal using the ANSI driver in z88dk. The initial testing of connecting to a BBS is working. There was a need to create the extended font set and then work out how to implement the extended fonts in z88dk. There is an interesting bug or not quite correct implementation of screen scrolling in ANSI drivers. ROM scroll is being used I think, which applies default attributes (black on white) to the new line. Problem with this is all BBS systems assume white on black, so you get nasty white lines feeding from the bottom on scroll. I managed to fix the scrolling line attributes, only to then find if a carriage return/line feed scrolls the screen. Again using the ROM code. So the dirty attribute bash I did using Ivan's ASM routine to paint attributes needs extending! Life is changing, life is busy! Emailed Thom my code for safe keeping, he seems to have managed to get the code accepted in to Z88DK! Guess that means I infect every project that uses the serial port on IF1 from now on!
The Speccy kids should get the Jetset Willy comments, only regret is not sticking Make-A-Chip in there... Well there is always the clean up version I need to make. https://github.com/z88dk/z88dk/blob/master/libsrc/rs232/spectrum/if1/rs232_get.c https://github.com/z88dk/z88dk/blob/master/libsrc/rs232/spectrum/if1/rs232_put.c I doubted I would be able to. Nothing like proving yourself wrong.
Managed to take the disassembled ROM code. Remove the offending BREAK routine and Border flicker. Plato works perfectly. So, I have not only had some C code published. I have now written a library/driver, with some assembly in it. Probably to be cleaned up and accepted by a compiler maintainer. Still can't believe the silly buggers put a check on the break key in there, and a half check at that!!! Did you miss me? Who am I kidding no one reads this, I will be lucky if Weebly is around when I am old and losing it. (When I will need to look back at these entries and remind myself of this time in my life.) I write for the enjoyment of it, like a graffiti artist spraying on a subway wall. This is me marking the world in some way. So what have I been doing, well more Plato PLATO PLATO stuff on the speccy. Having refurbished the old machine, checked the Interface1 is "safe", made an RS232 lead up. What do you think I did. I loaded the PLATOTERM up and connected it to IRATA.ONLINE. Bloody worked first time, got the lead correct the code worked every thing went well. Up until I decided to write a note to claim dibs on 48K on-line. Pressed space and the program stopped. After a few repeated attempts, the pattern of the problem wasn't a coincidence. You have some room for doubts with a machine over 30 years old, an interface of similar age with no knowledge of history. Touching a Sinclair Spectrum and having it randomly crash is not unusual behaviour many people have been known to tape EVERY thing down on the table to prevent crashes. Space on the old rubbery key 48K is a special key, it doubles as the BREAK key. Usually in combination with the Caps Shift key, to invoke the BREAK function. Looking at my modern keyboard its labelled as Pause, if you have one on your keyboard its probably dusty but in pristine condition. Its rarely used by most people, there is probably one person that will read this and say "I use it all the time". Great, what for?! Traditionally Break was the big red button for stopping programs that are executing. In today's multitasking multi threaded applications its a little difficult to know what you want to break! So knowing a little history, spotting the blanking (black ink on black paper) at the bottom of the screen it eventually clicked. Hitting space which I didn't need to do signing on, selecting menu items etc was causing the Break to fire. OK so why do we get a break event on a 48K and not on the +3. Thom's cool head got me in to line, back to basics he sent over 2 programs. One a key code print program, easy press key get 0x## hex code for the key. The second program was a tiny terminal, basically read a key send the key check for RS232 inbound data. Nothing exciting about the key codes. Really didn't take long to find the fault was still present with the tiny terminal. In a way its nice to know the main PLATO application wasn't at fault. After some playing about with tiny terminal it became apparent the break key was detected if Space/Break was pressed during and RS232 operation. Having looked at the fairly empty library files and Z88DK dev team confirming what was suspected. The library files are just wrappers for the ROM code routines. Z88DK developers gave Thom a bunch of tricks to try and clear the keyboard buffers etc to avoid the problem. I suspected the problem would inevitably be the ROM code itself, based on gut/experience. (Sods law, Murphy's law its that one place we don't want the problem to be!) The tricks worked well enough that I managed to post a note. It went from 100% failure on pressing space to 1%, you have to try to break out but it can still happen. Never expected to ever have a reason to look through the ROM disassembly of the spectrum. Surprising how the simplicity and almost beauty of assembly comes back to you. Nice to see comments in code to! Slapped boldly just before the bit banging of the line for data transmission, a call to check for "Break key". ...Sigh... Not a full check for break and Caps Shift, no just a quick cheeky check on the one key. Why would you do that?! From what I can tell, the RS232 port was only really intended for use with printers (back in the day serial printers were a thing). The ROM routines had been written with this in mind, hitting space/break would let you quickly stop a print job. The CPU bit bashing the port would lock the machine in to a fairly tight loop, this was the quick safety feature to stop printing. A design feature that would have stopped any one using basic for writing even a simple terminal application. If Clive Sinclair really wanted his machines to hit the Business market terminal applications would have been a big selling point. Granted there were modems devices sold, but a basic serial port enabled machine opens doors. Its impossible to know what was going through someones head at the time. May be the code was rushed known to be buggy so they just sold it as to be used for printing. Looking at the disassembly of the IF1 ROM thankfully its well documented with comments. Finding the serial routines was not difficult, commenting them out wont be difficult. Getting the entry and exit conditions well that might take a little work. ;; REC-BYTE One of the fun things about doing this sort of thing is leaving little Easter egg comments in the code. Some day it might snake its way upstream and sit in code that someone reads through looking for answers. It might just make someone smile at a time when they really need it! (P.S. Sorry if this code has screwed your project up!) ; OUT ($FE),A ; Change the border colour. - No Thanks Jetset Willy change the borders in program if wanted Hard part is trying to get the ASM code to start and end as required so not to crash. I have a lot of crashing going on, which is nice because I am changing things and it is having an effect on the system. Keep at it and we will see how long this takes...
Fitted the membrane for the keyboard. Had a problem with the keyboard randomly going from working to totally dead. From what I can make out the tracks on the membrane shorted out on the shielding on the video output. A dash of insulation tape over the metal parts confirmed it. I moved the tape to the membrane to be sure it won't catch anything else. Both ribbons insulated and the case closed, can't help but wonder if that was the problem with the original. The old membrane is packed away safe, it might be working but I am not up for any faffing today. Still not feeling great.
Interface 1 tested and working, did a little scrubbing on the edge connectors with white vinegar. Turned the queue tips from which to dull grey, so something has been removed. The IF1 seems to work, that is the basic interpreter works still on the spectrum. Usual childish 10 print "profanity" 20 go to 10 test program worked. Need to work on the RS232 cable pin outs. Hopefully better tomorrow and I can get the soldering iron out and get the leads made up. Then get on Irata.online, oh to be the first again would be nice. Issue 1 48K meets the PLATO over the internet. Feeling like crap today. What started as a throat infection kept me up with a cough, today is aches and pains.
Found randomly dumped in the porch was a jiffy bag though with the IF1. I've had a quick look over and clean of the connectors this evening. It has been fixed in that Sinclair style, so hopefully it works. |
AuthorMe, slightly crazy engineer type. Generalist in nature, hardware or software with nonspecialist skill set. Archives
January 2020
Categories
All
|