HyperTIES Markup Language
Simon Forman's stuff about xerblin is fascinating, and I'm excited about where it's heading, and how we can incorporate ideas from eToys into Python! I like the idea of having visual meta-languages that are compiled into Python, which avoids the problems of editing Python text or parse trees directly, and can support simplified "kindergarten" languages as well as more advanced forms.
The drag-and-drop stack and code outliner ideas work well with PostScript, which is a stack based but lispy code=data dynamic language that easily supports smalltalk-like object oriented programming via PostScript's "dictionary stack". Python + Cairo is also a great platform for implementing that kind of stuff, with dynamic layout of hybrid text and outline graphics, which scales and zooms and supports direct manipulation of data structures!
Here's a paper about PSIBER (PostScript Interactive Bug Eradication Routines), a visual interface to the PostScript interpreter in NeWS, and some links to video demos, too. Sorry about the flashing and poor compression -- they're recorded off a hires Sun monitor whose refresh rate was different than the camera, and I mercilessly compressed them a few years ago when the Internet was slower.
The Shape of PSIBER Space:
PostScript Interactive Bug Eradication Routines
PSIBER Demo: (9434433 bytes)
Demo of the NeWS PSIBER Space Deck. Research performed under the
direction of Mark Weiser and Ben Shneiderman. Developed and documented
thanks to the support of John Gilmore and Julia Menapace. Developed and
demonstrated by Don Hopkins.
One problem with PSIBER was that it was too easy to make a mistake dragging and dropping, and accidentally totally hose the internals of the window system, since you were editing shared structures in the NeWS server, like classes and canvases and event handler threads! It needed some kind of read-only safety shield or edit mode switch. Like Emacs, its main purpose in life was to develop and debug itself (and secondarily other NeWS applications like HyperTIES and NeMACS)!
A regular hierarchal outliner like most gui toolkits support might be too limiting for a visual programming language. Objects might have several ways to "open" them, and links coming in as well as going out. Any object might be at the intersection of several trees or sequences at once (like the class hierarchy, and the window hierarchy, the set of instances of the same class, and an ordered list of search results).
PSIBER supported "peripheral views" that let you attach embedded visual editors and open objects in different ways. Good XML editors support a branch for element attributes as well as a separate branch for sub-elements and text. Check out the way 3D Studio Max has outlines with two kinds of branching at each level of the 3d object tree (one branch for animatable object properties, and another branch for attached sub-objects), and the way it crosses a vertical outline with a horizontal timeline. It would be nice to be able to view an object in one or more hierarchies or sequences at once (like 3dsmax's property/sub-object outline + timeline), and easily pivot the editor between different hierarchies and sequences and alternative views (narrowing it to just a timeline, or just a sub-object outline, or a free-form graph view).
I can't remember what he called his system, but Steve Strassmann did some cool stuff on Mac Common Lisp or Dylan with "butterfly diagrams" that branched out in different directions, left for incoming links and right for outgoing links.
The closest thing I could google about Strassmann's butterfly diagrams
was his infamous "Is There Toscanini's Ice Cream in Heaven?" flowchart:
Marc H. Brown and Robert Sedgewick at Brown University developed a cool visual interface to Pascal called Balsa (named after a tree, of course), which supported multiple synchronized views of Pascal programs (lexical structure outline, Nassi-Shneiderman flowcharts, dynamic scope views, pascal syntax graphs, algorithm animation, etc). But it was pretty restrictive and ungainly about how you could input and edit a program (you could not do anything that wasn't syntactically correct, and I don't think it supported drag-and-drop), so you couldn't just type Pascal code into a text editor and watch the code views update in real time.
Here's a paper by Brad Myers that mentions Balsa and lots of other cool stuff like Henry Lieberman's "Tinker" Lisp programming by demonstration system:
Brad Myers: Taxonomies of Visual Programming and Program Visualization
Marc H. Brown, Robert Sedgewick: A system for algorithm animation
Henry Lieberman: Tinker: A Programming by Demonstration System for
Beginning Programmers, in Watch What I Do: Programming by Demonstration,
Allen Cypher, ed., MIT Press, 1993.
One problem with editing programs as text while trying to maintain a visual representation, is that typing in and editing a program as text puts the program through many syntactically incorrect states, before you've closed all your parens and balanced all your blocks, and you have a horrible correspondence problem mapping between changes in the text to changes in the structure. So it's hard to have your cake and eat it too. Even Emacs Electric-C Mode can get pretty annoying when it tries to close your parens and reindent your program for you while you're typing, if you're not trained to expect it. Of course it's much easier to attempt with languages like Lisp and Python that have simple clean syntax, rather than languages like Perl and C++ with complex byzantine syntax.
PS: Some weird videos:
Here's an incomprehensible video I recorded late at night, of the freaky
"PseudoScientific Visualizer" stuff:
Pseudo Scientific Visualizer Demo: (21431618 bytes)
Demo of the PseudoScientific Visualizer and NeWS PSIBER Space Deck.
Research performed under the direction of Mark Weiser and Ben
Shneiderman. Developed and documented thanks to the support of John
Gilmore and Julia Menapace. Developed and demonstrated by Don Hopkins.
HyperTIES Demo: (3562242 bytes)
University of Maryland Human Computer Interaction Lab HyperTIES Demo.
Research performed under the direction of Ben Shneiderman. HyperTIES
hypermedia browser developed by Ben Shneiderman, Bill Weiland, Catherine
Plaisant and Don Hopkins. Demonstrated by Don Hopkins.
NeMACS Demo: (3511315 bytes)
Demo of UniPress NeMACS running in the NeWS Window System. Emacs
development performed under the direction of Mike Gallaher. NeWS user
interface developed and demonstrated by Don Hopkins.
HyperLook SimCity Demo: (49816346 bytes)
Demonstration of SimCity running under the HyperLook user interface
development system, based on NeWS PostScript. Includes a demonstration
of editing HyperLook graphics and user interfaces, the HyperLook
Cellular Automata Machine, and the HyperLook Happy Tool. Also shows The
NeWS Toolkit applications PizzaTool and RasterRap. HyperLook developed
by Arthur van Hoff and Don Hopkins at the Turing Institute. SimCity
ported to Unix and HyperLook by Don Hopkins. HyperLook Cellular Automata
Machine, Happy Tool, The NeWS Toolkit, PizzaTool and Raster Rap
developed by Don Hopkins. Demonstration, transcript and close captioning
by Don Hopkins. Camera and interview by Abbe Don. Taped at the San
Even more weird videos: http://www.donhopkins.com/home/movies/
Designing to Facilitate Browsing: A Look Back at the Hyperties Workstation Browser
By Ben Shneiderman, Catherine Plaisant, Rodrigo Botafogo, Don Hopkins, William Weiland.
Human-Computer Interaction Laboratory
A.V. Williams Bldg., University of Maryland
College Park MD 20742, U.S.A.
Since browsing hypertext can present a formidable cognitive challenge, user interface design plays a major role in determining acceptability. In the Unix workstation version of Hyperties, a research-oriented prototype, we focussed on design features that facilitate browsing. We first give a general overview of Hyperties and its markup language. Customizable documents can be generated by the conditional text feature that enables dynamic and selective display of text and graphics. In addition we present:
- an innovative solution to link identification: pop-out graphical buttons of arbitrary shape.
- application of pie menus to permit low cognitive load actions that reduce the distraction of common actions, such as page turning or window selection.
- multiple window selection strategies that reduce clutter and housekeeping effort. We preferred piles-of-tiles, in which standard-sized windows were arranged in a consistent pattern on the display and actions could be done rapidly, allowing users to concentrate on the contents.
HyperTIES is an early hypermedia browser developed under the direction of Dr. Ben Shneiderman at the University of Maryland Human Computer Interaction Lab.