Saturday, May 9, 2015

Esp8266 Esp-12

There are several configurations of the Esp8266 modules which is denoted with a Exp-XX suffix. Currently I know of Esp-01 through Esp-12, each have their advantages and dis-advantages.

Although I have be experimenting with the Esp8266 Esp-11 (see previous post), my goal is to use the Esp-12 configuration for more complex applications. The Esp-11 configuration has only 8 pins, while the Esp-12 has 16 pins, and therefore more I/O options.

Breakout boards are available for each, but there are trade-offs; the Esp-12 Breakout Board is much wider than the Esp-12 itself. Even though the breakout Esp-12 Breakout Board is protoboard friendly (i.e., 0.1in spacing), it is so wide that there is little space to attach jumpers (I think only one hole row on each side is usable).

I decided to try the technique of bending header pins as necessary to match the 2mm spacing, I originally used this method on the Esp-11 (see previous post).

The pin spacing of the Exp-11 is smaller at 1.27mm, while the Esp-12 is 2.0mm. A good microscope is needed to solder headers directly onto either.

Esp8266 Esp-12 with two 0.1in Headers Attached

Now all that is needed is wiring-it-up and some software (Yeah, . . . famous last words).


--

Friday, May 8, 2015

Esp8266 - Programming Success

For several weeks, I have been failing at programming the Esp8266 ES-11.

I had purchased a group of five Esp8266's for initial experimentation. I also purchased a small number of 3.3v/5.0v FTDI (USB to Serial) Adapters which are used to program the Esp8266's (see previous post).

Esp8266 and FTDI
The FTDI is also used to configure the Esp8266's using simple (default) "AT" instructions.  I could configure the Esp for "Station", "Access Point", and for "Both" modes. In "Both" modes, the Esp can be configured as a WIFI-to-WIFI Router. The cellphone APPs; "Net Scan" and "WIFI Analyser" are very useful to understand what is going on (or not).

Everything was looking great.

So, the next step was to compile a program, to replacing the Default functions with a Program that I have written. The instructions looked simple, and there are several Programming Environments to choose from; Arduino IDE, nodemcu, ESPlorer, and others. Besides, supposedly, there are easy to use programs to re-Flash the device back to the "Factory Fresh" original (ROM) state (so I thought).

This is where the Trouble Started.

At each attempt to do something interesting, that is; Program the Device or Reflash it back to original configuration - I killed another of my Esp8266s, in other words I BRICKED it!

Of the five purchased Esp8266's, I managed to BRICK three of them, I only had two good Esp's left. Now, being gun shy, with the new Exp8266, everything was suspect.

The goal became "How to UnBrick a Esp8266" without "Bricking" another. I have been working this problem for about two weeks without progress or solution.

There is a ton of information on the Internet regarding Bricked Exp8266's, but a lot of it is confusing and conflicting.

This post is about what I did to UnBrick my Esp8266's.

I gave my friend Jeff - KO7M a Bricked Esp8266 and one of my FTDI USB-to-Serial Adapters. We worked several days trying to download or ReFlash the Exp8266 - without much luck.


The Fix.

After several more hours of work, Jeff suggested and tried a different FTDI.

It worked !

To make a very long story short, we replaced the FTDI with another variety, the Parallax PropPlug. We think the previous FTDI was corrupting the downloaded data that was being sent to the Esp8266's.

Yes I know, the PropPlug is a 5V device and Esp8266 are 3.3v devices. But I forgot that on the original test, and it is working, so I have not changed the configuration (only TX/RX/RST/GND are connected).

The PropPlug (on the right)
connected to
the Esp8266 (on the left)
Question: why would the original FTDI work interactively, but would NOT work to download Batch and Binary files? - "I Don't Know, and Don't Care" - with the PropPlug, I now have something that works.

Since I started using the PropPlug I have NOT had a single failure, and we have recovered each of the Esp8266's with either the original Binaries, or my compiled programs.

Currently I am playing with Arduino IDE Compiled WIFI WebServers, one of the servers has been running for greater than 48 hours, and has serviced about 4800 test page requests. I have three running side-by-side.

Three Web Servers
These will become a part of my Internet-of-Things (IoT).

I can't guarantee that they are online, but you can check at: http://dc02.ebcon.com:8160/

Esp8266 Web Server
Note: the page refreshes each 30 seconds.

More fun and information to follow.

--

Monday, May 4, 2015

K7QYP - A Bit of History

 A Bit of History, and a Blast from the Past
(the way I remember it)

About 48 years ago, I graduated from the US Navy's "A", "B" and "C" Electronic Training Schools. Due to the Training received in "C" School, also known as "PMEL" (Precision Measurement Electronic Lab), I was assigned to the Electronics Calibration Lab aboard the USS Howard W. Gilmore AS-16 in Charleston SC. Where I met Don Seahulster - K7QYP, who was my Shop's Lead.

Don was a Amateur (Ham) Radio Operator. On our first trip to sea together, Don brought his Ham Gear and portable antenna on board. The antenna was mounted on the hand rail outside the shop door. Don worked many Phone Patches back to the States for crew, including the XO which enjoyed getting updates and weather from his wife. The FCC, the Captain, and the Shipboard Radio Operations Officer were all aware of and supported Don's Ham Radio activities. To be able to talk with folks at home was a BIG morale boost for the Crew!

I was not a Ham at the time, this activity convinced me to get a Ham Ticket.

One of our planned Port-of-call was San Juan Puerto Rico, which just happen to have a FCC Field Office. Bob Stothfang and I studied together in preparation for the Ham test, we were both just out of Naval Electronic School and therefore hopefully, it would be easy, we just needed to brush up on the Regulations and some radio details.

At the FCC Office we both took the Amateur Radio; Technician, and General Tests. We also took the FCC; Third, Second, and First Class Radio Telephone License Tests. And while we were there, we took the Ship Board Radar Endorsement Test. A few weeks later, Bob and I, both received notice that we had passed ALL tests  :-)

I became; Eldon - WA0UWH, and Bob - WB8BEQ.

Later while on board, we were in the Ship Yard for some Refit and Repairs. Don received a new piece of Lab Equipment, a HP-117A VLF WWVB Receiver. We were the Shipboard Calibration Lab and this was to support the Time Standard for all calibrations. Of course, the VLF Receiver needed an outside antenna mount, and with the best coax money could buy, all plumbed directly to our LAB.

The Ship Yard did a great job on the installation, all of the through bulkhead water tight fitting were first class. The VLF Loop Antenna mount was up on the HeliDeck, but within arms reach while standing on the deck hand rails, this was so that the loop antenna could be rotated as necessary at each Port-of-Call (or at least that is what Don told me).

On our next trip to Sea, Don again brought his Ham Gear, and a new AVQ Antenna, which we mounted on the HeliPort VLF Antenna mounting stub (removed VLF antenna), with direct coax cable access to our LAB. With this configuration, we had much better signal into the States,  which produced many more and better Phone Patches for the Captain, XO and the Crew.

So, why relate all of this history?

Well because, a piece of that History has come full circle.

One of the QSL cards that Don sent 47 years ago, while on board the USS Howard W. Gilmore AS-16 running Phone Patches for the crew, was just recently seen on ebay for sale. And yes, Don has purchased it.

K7QYP - QSL on Ebay

Note, the 4 cent Stamp
Don sent the card for 4 cents, and received it back 48 years latter for $12.00 plus shipping - now that IS inflation !

When Don left the Navy, I became the Calibration Lab Lead, and continued the tridition of providing Phone Patches for the Crew with my Ham Gear and a 4BTV mounted on the same VLF antenna mounting stub.

Don is no longer K7QYP, he is now W4LSC, I am still WA0UWH, and I think, Bob is still WB8BEQ?

Somewhere, I still have the Radio Logs of those calls.

All Good Memories :-)

--

Friday, May 1, 2015

Enigma - Hardware Implemented in Software

For several years, actually more like 30, I have being fascinated with the working of the Enigma Machine.

On several occasions I have toyed with a Enigma Machine software design, but was never able to build an accurate solution. I have a very specific implementation in mind. So far, nothing has worked correctly. But, I guess, I have had a low-grade mental effort constantly considering the problem.

About three years ago, Wardy posted his build of a hardware implementation. That renewed my interest, but even with more work on my software implementation, and even with still a lot more thinking, there was no real progress.

Recently, my friend Jeff - KO7M posted his Enigma Machine implementation, I did NOT read his details, because I wanted to continue with my implementation without external insight. But then again, his efforts renewing my interest in my software implementation. I have a specific implementation in mind that I wanted to pursue.

So now, after several weeks of casual work on my software Enigma Machine, it still does NOT work as expected, some character are encoded correctly, but many others are NOT.

So, . . .

At night, I wake up,  thinking about the problem. And each night I seem to understand a little bit more.

There are several ideas that have helped me understanding the Enigma Machines operation. To simplify, I like to think of it in terms of:
  • Forget the Plug Board, it is a simple character exchange.
  • Code Wheels act like a "Fixed Wired, but Rotating PlugBoard", with simple two-letter exchanges.
  • The logic of the rotation of wheels is easier to understand if I think about it like a PlugBoard without cables plugged in, that is, character are not exchanged (i.e., A-to-A, B-to-B, etc)
  •  And, the Reflector is a simple adjacent character exchange (i.e., A-to-B, C-to-D, etc)
  • The code algorithm from the Input-to-the-Reflector, and the Reflector-to-the-Output, should mirror each other.
  • All wheel advancement and algorithms use Clock Maths.
  • Exchange of characters can be implemented after a correct wheel advancement algorithm is found and implemented.

Last Night

Last night, I suddenly woke up, . . . and had a algorithmic solution I wanted to try. In about five minutes, I had it typed in (see below).


IT WORK!

With proper Settings, I can now correctly Encode and Decode any M3 and M4 Enigma message, as verified.

ABC AAA AAA --
VERYXCOOLXJEFFXCANXYOUXREADXTHISXX
RLXVTMMQARSTKDJADSKMLAPGCXBRPZZFPE

The following is the guts of my software Enigma Machine implementation:



# Get Message Character
$i = ord($m)-65;

# Inc Wheels as Necessary
@P[3] += 1;     # Increment with each character
@P[2] += ((@P[3]%26) == $Adv3_Static+1);
@P[1] += ((@P[2]%26) == $Adv2_Static+1);
@P[0] += 0;

$R3 = @P[3] % 26;
$R2 = @P[2] % 26;
$R1 = @P[1] % 26;
$R0 = @P[0] % 26;

# Do Enigma, Encode or Decode
$i = @PB[$i];
$i = (@WL3[($i+$R3)%26]+26-$R3)%26;
$i = (@WL2[($i+$R2)%26]+26-$R2)%26;
$i = (@WL1[($i+$R1)%26]+26-$R1)%26;
$i = (@WL0[($i+$R0)%26]+26-$R0)%26;
$i = @R[$i]; 
$i = (@WR0[($i+$R0)%26]+26-$R0)%26;
$i = (@WR1[($i+$R1)%26]+26-$R1)%26;
$i = (@WR2[($i+$R2)%26]+26-$R2)%26;
$i = (@WR3[($i+$R3)%26]+26-$R3)%26;
$i = @PB[$i];

# Show Light, The Encode Character
print chr($i+65);



I will post a complete listing, after I code a proper User Interface and implement Multi-Notch Wheel advancements.

There are some details of newer Enigma Machine versions that I will not try implement, but they should be simple.

Note: There are probably optimizations that could be used in the code (I am still learning python).

This has be a long, mentally challenging, and fun project. Now I need to find another  :-)

--

Tuesday, April 28, 2015

Esp8266 - Easy Programming - NOT! (or, at least not yet)

UPDATED: see below

I have been playing with my Esp8266 (see previous post). I received these a few weeks ago, but before use, I had to wait to receive a 3.3 Volt USB to Serial Programmer.

Esp8266 Wired and Ready
To work easily with a Protoboard, I removed the original angled header from the Programmer, and replaced it with a "straight down" header. Unsoldering the header was a pain, because it was assembled with Lead-Free Solder, which does NOT melt very easy - I hate that stuff ! - (A note to the Manufacture, the EPA, and OSHA, I was not planning on eating or licking the board, normal 63/37 solder would have be fine  :-)


WIFI Analyzer
Once Wired, the Fun Starts.

Out of the box (the bag actually) this little board seem easy and is interesting to set up. With the help of online resources I was able to send and receive messages from the serial interface.

The supplied "AT+" commands allow you to change the WIFI mode and connect to a local WIFI Access Point.

I use a Cellphone App; "Wifi Analyzer" to see the Esp8266's RF output, along with other local WIFI AP's. 

And, another App; "Net Scan", was used to check for the IP Address as seen from my network.

The ElectroDragon.com web page provides some good instructions and examples, but the way the pages is written as an Image, you can NOT Cut-n-Paste from it, now how dumb is that!

The following is an example of a normal initial dialogue with a Esp8266, recorded here for my future reference (Cut-n-Paste works here). Note: to make the listing shorter, some blank lines have be removed.




AT
OK

AT+RST
OK
c_�� #��FjS�fJ;��
[Vendor:www.ai-thinker.com Version:0.9.2.4]
ready

AT+CWMODE=3
OK

AT+CWLAP
+CWLAP:(4,"EBCON",-41,"00:16:01:84:c4:a8",4)
OK

AT+CWJAP="EBCON","mypasswd"
OK

AT+CWJAP?
+CWJAP:"EBCON"
OK

AT+CIPMUX=1
OK

AT+CIFSR
192.168.4.1
192.168.2.164
OK




But, NOT all is well in Paradise.

I have managed to "Brick" the first two of my Esp8266's (only 3 more left).

After the initial set up and tests, my desire was to install an application onto the Esp8266, I chose a simple example "Hello World - Web Server" from the specially configured Arduino IDE 1.6.1.

The download (actually Uploading) went as expected, with a status of "Done Uploading". But, that is when the troubles began. I can no longer see the RF output on the WIFI Analyzer, the IP Address does not show up on Net Scan, and the RESET status text sent to the Terminal Window is just Binary Trash, regardless of the Baud Rate or Terminal Settings that I choose.
Binary Trash

Update: With help from Jeff - KO7M, and his Digital Analyser we have determined the "Trash" is a "Reset Message" sent at 74115 Baud (or 74880 baud nominal), most of my usual programming tools do not have that baud rate as an option. But even when forced, no joy:




$ ./esptool.py --baud 74880 -p /dev/ttyUSB* write_flash 0x000000 foo.bin
Connecting...
Erasing flash...
Writing at 0x00000000... (0 %)
Traceback (most recent call last):
  File "./esptool.py", line 538, in <module>    
  esp.flash_block(block, seq)
  File "./esptool.py", line 197, in flash_block
    struct.pack('<IIII', len(data), seq, 0, 0)+data, ESPROM.checksum(data))[1] != "\0\0":
  File "./esptool.py", line 106, in command
    raise Exception('Invalid head of packet')
Exception: Invalid head of packet



We were able to extract binary from a known "good Esp8266", but was unable to flash a "bad Esp8266" with that binary (see above).

Currently I am, "still" Lost.

There is a lot of conflicting information regarding; "How to UnBrick a Esp8266", the suggested programs do not work well with Linux, and the Firmware files were very hard to find. The most helpful online resource seems to be from this blog.

But, so far, I have NOT been able to "UnBrick" my two Esp8266's, they are cheap and I could just toss them, but I want to figure this out.

Here is what I see on a Workstation terminal, note: I am in group "dialout" and have normal write access to /dev/ttyUSB0. I thought that maybe the Serial Data Rate was wrong (see above), but then I can repeat the Arduino re-program just fine, with the proper finished "Done Uploading" report.


$ ls -l /dev/ttyUSB*
crw-rw---- 1 root dialout 188, 0 Apr 28 17:03 /dev/ttyUSB0

$ ./esptool.py -p /dev/ttyUSB0 write_flash 0x000000 AI-v0.9.5.0_AT_Firmware.bin
Connecting...
Traceback (most recent call last):
  File "./esptool.py", line 473, in <module>
    esp.connect()
  File "./esptool.py", line 151, in connect
    raise Exception('Failed to connect')
Exception: Failed to connect




I am hopeful, . . . and when I do find the answer, I will update this post with additional information.


UPDATE: It works, see post.

--

Sunday, April 12, 2015

More Playing with MQTT

With updated software, mosquitto version 1.4.1, it is very easy to connect two Brokers (see problem as suggested on previous post). A few new line in the configuration files is all that is needed. To publish (upload-only) my Broker's Topics to the public Broker at iot.eclipse.org, I used the following.

Content of: /etc/mosquitto/conf.d/bridge.conf


connection ebcon.com
address iot.eclipse.org
topic # out 0 "" ebcon.com/



Out-bound topics have "ebcon.com/" pre-pended, currently I do not allow or receive data from iot.eclipse.org directly by my Broker. If I want a topic, I subscribe directly with an application (mosquitto_sub).

My own local Brokers (in three different physical locations) exchange data from the two lesser Brokers in bidirectional fashion. Brokers do not have to be hierarchical, they can be peers. In my case I chose to configure the "bridge" on what I consider the lesser Brokers.



connection remote1
address <remote broker IPA>
topic # both 0



As a test, I created a "JoyStick" control publisher, when the joysticks are moved, all Brokers receive the data, much faster than I expected. Performance is very good. Eventually, I will control remote servos to direct (point) Raspberry PI Cameras.

You can subscribe to my testing with:



$ mosquitto_sub -v -h iot.eclipse.org -t 'ebcon.com/#'




Note: Mosquitto MQTT Brokers can publish any kind of data, including Web Pages and Photos.



I am just learning and having FUN with MQTT !

--

Esp8266 With 0.1 Inch Header

In preparation for programming my Esp8266 and the Internet of Things (IoT), I have installed a 0.1 inch header. The connector pad on the Esp8266 are 2mm and therefore a lot of lead bending was necessary.

I think this was my first soldering project that required full 15X Microscope magnification. The Esp8266-Esp11 board is much smaller than implied from the ebay photos (a previous post). The on-line diagrams and documentation are correct, but in your hand it is very small.

Esp8266-Esp11 with
0.1 Inch Header Installed
I have on order the programming interface, which should be received soon. I will use the Esp8266 with the installed header for use on Protoboards, actual projects will use direct solder to matting "Castellated Pads" on custom PCB's.
Esp8266-Esp11 Pin Layout

For my later reference, I have included the pin diagram.

--

Tuesday, April 7, 2015

MQTT - Internet of Things (IoT) TimeService

I am still playing with the Internet of Things (IoT) and MQTT.

I have implemented a simple TimeService, so each of my (future) devices can subscribe to-and-know the correct current time. This script is more complex than needed, but shown here as an example of what can easily be done with a simple script.




#! /bin/bash
# A mosquitto Time Server
# Author: Eldon Brown - eldonb@ebcon.com

    # Usage: progname [-v] [-p TopicPrefix] [-h BrokerHostAddress] [other mosquitto options]

    # Check for Verbose Debug Option
    case "$1" in
        (-v):; set -xv; shift 1 ;;
    esac

    # Set Topic and Service and then add "Prefix" to Topic if desired
    TOPIC="Time/`date +%Z`"
    SERVICE="client/${TOPIC}"
    case "$1" in
        (-p):; TOPIC="$2/${TOPIC}"; SERVICE="$2/${SERVICE}"; shift 2 ;;
    esac

    HOST='' # Assume Local Host
    case "$1" in
        (-h):; HOST="$1 $2"; shift 2 ;;
    esac


    mosquitto_pub -r ${HOST} -q 1 -t "${SERVICE}" -m 1 $*
    trap "mosquitto_pub -r ${HOST} -q 1 -t \"${SERVICE}\" -m 0 $*; exit 0" SIGHUP SIGINT SIGTERM

    while true
    do

        sleep `date '+(60-%S) %% 10' | bc` # Provide on 10 sec intervals

        date '+%s %Y/%m/%d %H:%M:%S'

        sleep 2          # To avoid multiple reports for the same second

    done |

    mosquitto_pub ${HOST} -t "${TOPIC}" -l $*

# End



For a while I will "publish" this time, as a service on a the public MQTT Broker, at: iot.eclipse.org

You can connect and see the results by executing:



$ mosquitto_sub -h iot.eclipse.org -v -t 'ebcon.com/client/#' -t 'ebcon.com/Time/#' 



The -t 'client/#' topic is retained (-r) by the Broker and will report that the TimeService is active (1), or not (0).

Just for fun, You can subscribe to everything on the iot.eclipse.org MQTT Broker, but be prepared to be Over Whelmed with data!



$ mosquitto_sub -h iot.eclipse.org -v -t '#' 




So, you ask: Just how does MQTT fit into my Ham Radio plans?

I plan to implement "remote control" of radios (among many other things) using MQTT.

More information to follow.

--

Saturday, April 4, 2015

MQTT - Bridge Work-a-Round

 UPDATE:
I was using a very OLD version of "mosquitto" which did not support Bridges well, I am hoping the current version will work better.

-

To experiment with the Internet-of-Things (IoT), while waiting to receive my Esp8266's (see previous post), I have enlisted my Raspberry PI's into playing the part of a remote data collection device. My Desktop Workstation is the Control Point and the Master MQTT Message Broker.

Originally I had planned to use a Master/Slave Bridged Broker configuration, where the Workstation would be the Master, and the Raspberry PI's would be a Bridged Slave Brokers, and they would collect data and messages from the Esp8266's.

But, I have not been able to get the "mosquitto" Bridged Broker configuration to work as documented :-(

I have a work-a-round; I will temporary use an "ssh tunnel" to directly collect messages from the Raspberry PI's.

At each Raspberry PI's, an "ssh tunnel" is created for "port 1883" (the MQTT port), and therefore the Master Broker is directly available to the Raspberry PI's. A side benefit of is; the "tunnel" is compressed and encrypted.

On Raspberry PI:


$ ssh -CnNTf -L 1883:localhost:1883 <master workstation IPA>




With this setup, I can conduct; tests for camera servo control, send SMS messages to the Cell Phone, and other experiments (see previous posts).


--

Friday, April 3, 2015

The Internet-of-Things and MQTT

UPDATED: 2015/04/07
To include mosquitto2sms status via mqtt

The Internet Hobby Electronics world is ABLAZE with Esp8266 excitement.  These Ebay $3.00 modules are being used to create some very interesting Internet-of-Things (IoT) projects. Search Google and Youtube for Esp8266.

To make interesting projects, code can be downloaded into the processor on the Esp8266, or it can be connected to a processor like the Arduino. In either case, access to the Internet for a Hobby Project is very exciting.

One way to provide status and/or control to/from the Esp8266 is via a (little known) protocol called "MQTT", which provides simple and small processor code footprint. Actually, it is not unknown, Facebook uses it !

For Raspberry PI and Linux people, there are downloadable commands that implement MQTT, the package that contain these are "mosquitto" and "mosquitto-clients"




$ sudo apt-get install mosquitto mosquitto-clients




With "mosquitto" and the "curl" command (see previous post), I implemented a mosquitto2sms gateway, as:

mosquitto2sms


#! /bin/bash
# Author: Eldon R Brown - WA0UWH

 # Usage: mosquitto2sms [-h BrokerHostAddress] []


 # set -xv   # For Debug

 mosquitto_sub -v -t 'ToSMS/#' $* |

 sed -u -n 's/^ToSMS\/\([0-9][0-9]\+\) /\1 /p' |         # Extract just the phone Number and Message

 while read phone mesg
 do

  # echo -e "\n\n${phone} ${mesg}"  # For Debug
  curl http://textbelt.com/text -d "number=${phone}" -d "message=${mesg}" 2>&1 |
  grep -q true &&
  mosquitto_pub -t "ToSMS/${phone}/Stat" -m 1 $* ||
  mosquitto_pub -t "ToSMS/${phone}/Stat" -m 0 $*

 done

# End




On another system, or any MQTT device, a simple message can be easily sent to a Cell Phone. Example as shown is from Raspberry PI.




mosquitto_pub -q 1 -t ToSMS/2025551212 -m 'Test Message'




To watch (or monitor) all of the activity on the messaging system, execute the following (i.e., subscribe to all):




mosquitto_sub -v -t '+/#'




 I am anxiously awaiting the arrival of my ebay Esp8266's, . . . only a few more days !

--