llc-suite (LBNL LLRF Control Suite) Larry Doolittle, Lawrence Berkeley National Lab, March 2004 llc-suite Copyright (c) 2004, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. Your use of this software is pursuant to a "BSD-style" open source license agreement, the text of which is in license.txt (md5sum a1e0e81c78f6eba050b0e96996f49fd5) that should accompany this file. If the license agreement is not there, or if you have questions about the license, please contact Berkeley Lab's Technology Transfer Department at TTD@lbl.gov referring to "llc-suite (LBNL Ref CR-1988)" This kit is centered around synthesizable Verilog code that targets the FPGA for the LBNL-derived SNS LLRF systems. It also includes test benches that exercise most of that code, and the HAL (hardware abstraction layer) that the host computer uses for access to the functionality of the FPGA. It targets all three generations of SNS LLRF hardware: (1) MEBT, (2) Interim, and (3) FCM. Generations (2) Interim and (3) FCM have been fully tested with hardware. Generation (1) MEBT presumably needs some debugging. This kit is standards-based. Most of the key intellectual effort is invested in IEEE 1364 Standard Verilog and ISO/IEC 9899:1990 Standard C with a POSIX.1 library. Of course, this is more of a goal than a reality, since the result is only checked with real and imperfect tools. The synthesizable Verilog code works with Xilinx XST 5.2i, and synthesizes without errors on Synplicity Pro 7.3.4 (functionality not yet demonstrated). All the Verilog code simulates under Icarus Verilog (most versions starting with the 20030904 snapshot should work, I personally follow CVS). The Verilog code is mostly Verilog-1995-compatible, but with two Verilog-2001 features thrown in: ANSI-style parameter lists, e.g., module foo( input host_clk, output [15:0] readout, output reg bar); and (for simulation only) signed wires+regs, e.g., reg signed [16:0] diff; always @(negedge clk40) $display("signed answer is %d", diff); Although support for the full Verilog-2001 language is hard to find, this subset is compatible with: Icarus Verilog s20030904 and later Xilinx XST 5.1i and 5.2i Synplicity Pro 7.3.4 Modelsim PE 5.8 Only the first two tools on that list have been field-proven. We (SNS) are working on exercising the next two. We now have some preliminary success with Modelsim on Linux: to try it, change a line in source/llrf.rules.make from "MODELSIM = no" to "MODELSIM = yes". The C code uses a compatible subset of C89 and C99, but with occasional use of the "long long" 64-bit data type (which is in C99 but not C89). It also uses POSIX and BSD library features. It has been successfully tested using the following gcc versions: gcc-2.95.2 (Solaris), gcc-2.95.3 (home-made, both for StrongARM and i386/RH5.2), gcc-2.96 (Red Hat 7.3), gcc-3.1 (Mac OS-X 10.3.2), gcc-3.3 (SuSE 8.2), gcc-3.3.1 (cygming special), gcc-3.3.3 (Debian sarge). At some point it needs to work under Microsoft Visual C/C++ in order to work with Modelsim, unless someone learns how to get the equivalent DLL file structure using Mingw. As a scripted, push-button ensemble, the kit has been successfully tested on the following systems: Red Hat 5.2 GNU/Linux (with upgraded gcc and make) Red Hat 7.3 GNU/Linux SuSE 8.2 GNU/Linux Debian sarge GNU/Linux MacOS X 10.3.2 Solaris 5.8 Cygwin on Microsoft Windows 2000 Professional The required Cygwin (http://www.cygwin.com/) packages for Microsoft Windows, beyond the base system, are perl, bc, gcc, and make. In the Cygwin setup menu, look under Interpreters, Utils, Devel, and Devel, respectively. At this juncture Microsoft support should probably be labeled "for experts only". Besides gcc and verilog, it obviously needs sh, gmake, perl, gawk, bc, grep, octave+gnuplot (only for the history buffer tests), and blt (only to run the demo GUI). In that environment, "sh run_all" in this directory will take a few minutes to run all the tests. If you adjust the paths in this script to find a cross-compiler, it can also build utilities (based on the HAL) and a toy control system for the StrongARM/nanoEngine. Scripted synthesis in the source directory, given a functional Xilinx WebPack 5.1i or 5.2i, should work under both Linux/WINE and Microsoft/Cygwin. Just type "make llrf_interim.XCV150.bit". A word to those who would browse the Verilog code: you will find a distinct lack of modules representing the top-level FPGA. All such Verilog code is generated as needed by source/intercon.pl, based on the component submodules. This applies to both the synthesized FPGA (llrf_interim.v) and all test benches (e.g., test-host/register_fpga.v). See the Makefiles and intercon.pl for more information on this process. Tests sorted by directory: source/ synthesizable code compiles measure lines of Verilog code test/ CORDIC DKCM totalizer test-host/ config ROM, shadow RAM, and feedforward RAM timer register reads and writes stopwatch ds1822 ds2401 bit-serial devices test-cavity/ close feedback loop with simple cavity model in Verilog, no host computer close feedback loop with cavity model in C, still no host computer close feedback loop with stand-alone cavity simulator in C, no Verilog test-history/ (visual rather than PASS/FAIL output) 12.8 microseconds someplace even 12.8 microseconds someplace odd 25.6 microseconds at end of pulse 25.6 microseconds in middle of pulse 25.6 microseconds at beginning of pulse (doesn't even fill buffer once) 102.4 microsecond buffer, shows whole pulse 332.8 microsecond buffer, shows mux and non-power-of-two normalization 1638.4 microsecond buffer, maximum allowed, rather boring output host/ decay curve fitting (without time skew) Some interesting documentation is in the doc directory. Items in graphical form are the files layers2.fig (shows the nested interrelationships between hardware and software), datapath3.fig (shows the numerical computations performed in dds.v and fdbk_loop.v; blue highlighted boxes are registers, with names that match the Verilog code), and cosimulation.fig (see commentary in cosimulation.txt). These figures are in FIG 3.2 format; xfig is free software, easily available on Linux, Unix, and Microsoft/Cygwin. Almost everything here is freshly written software by Larry Doolittle , 510-486-7382 (for more contact information, see http://recycle.lbl.gov/~ldoolitt/). Exceptions and acknowledgments: ** test-host/cordic2.c is Copyright (C) 1981-1999 Ken Turkowski, published under a free license; see that file for details. ** test-host/{messagev.c,fifo.c,nanoengine.v} are based on ideas found in Steve Williams' pci-sim package. The code itself has been rewritten, and so is not derived from that code base. I have an e-mail on file in which Steve agrees with that conclusion. ** source/cordic.v started in collaboration with Ming Choy, while he was a summer student at LBNL. No meaningful traces of his original code remain. ** source/fdbk_loop.v started in collaboration with Justin Oo, while he was a summer student at LBNL. No meaningful traces of his original code remain. ** source/{BUFG.v,IBUF.v,RAMB4_S4_S4.v,RAMB4_S8_S8.v,SRL16E.v} are necessarily interface-compatible with the Xilinx unisims versions. These have been rewritten both for redistribution convenience and for simulation speed, dropping features that are not used here. For size and/or license reasons, this package does _not_ include the following: EPICS (see http://www.aps.anl.gov/epics/) Software connection from toy server to tuning device Circuit schematics or board layout nanoEngine (or other host) operating system (see http://recycle.lbl.gov/~ldoolitt/bse/) Linux driver to connect the FPGA interrupt pin to a device file Windows-compatible socket-based transport layer for host simulations Program to download FPGA bit file to the chip Simulation of VXI interface Hardware ;-) I leave you with this quote: Programming today is the opposite of diamond mining. In diamond mining you dig up a lot of dirt to find a small bit of value. With programming you start with the value, the real intention, and bury it in a bunch of dirt. -- Charles Simonyi http://www.technologyreview.com/articles/tristram1103.asp?p=0 I actively search out creative ways to bury the valuable functionality of my programs in as little dirt as possible!