2. Literate programming

Today's topics

Literate programming

Markup languages

Macro programming

Text formatting system

Runninb programs on Scrapbox

Create your account on Scrapbox for taking the course.

You need a Google account (for GMail, etc.) to create a Scrapbox account.

You can create a Google accout here.

You can also use your keio.jp account (G suite account).

You can create a Scrapbox account at https://Scrapbox.io/ using your Google account.

Create a new project sfc-pm2017-(loginID) using your student ID.

e.g. /sfc-pm2017-s00000tm

Use this page for all your activities like class attendance notification, paper submission, asking questions, etc.

Set the page to the "public" mode.

Click "Settings" in the menu at the top-right corner.

https://gyazo.com/4bb8c1fc6a8eee19461f63d69f7bbaa6

Literate programming

Proposed by Donald E. Knuth in the 80's

Integrate codes and related texts into one document

Codes, manuals, test scripts, etc.

Treat codes and documents like a “literature”

Pretty-printing (beautiful typesetting)

Knuth's Book on “literate programming”

https://upload.wikimedia.org/wikipedia/en/6/62/Literate_Programming_book_cover.jpg

Donald E. Knuth

Famous professor on computer science

Stanford university

Inventor of many algorithms

Author of many computer books

Author of TeX

TeX

METAFONT

Donald E. Knuth

http://gyazo.com/9f7e6185fadd5589d903fe47cb45e4f9.png

TeX

Developed by Knuth around 1980

Macro programming language

Markup language for text formatting

code:test.tex

\def\name{増井}

My name is \name.

The TeX Book (1984)

http://gyazo.com/84e7403307e75b2f6487c832dc59e14e.png

Many papers have been written in TeX

Macro programming

String substitution by 'macro processor'

e.g. #define in C

sample.c

code:sample.c

#define ELEMENTS 100

int aELEMENTS;

#define max(x,y) ((x)>(y)?(x):(y))

max(func(a),func(b))

cc -E sample.c

code:cc

# 1 "sample.c"

# 1 "<built-in>"

# 1 "<command-line>"

# 1 "sample.c"

int a100;

((func(a))>(func(b))?(func(a)):(func(b)))

sample.html

code:sample.html

<html>

<head>

<title>__system__について</title>

</head>

<body>

__system__というシステムについて解説します。

<br>

__system__は、2013年に...

</body>

</html>

cc -D__system__=ABC -E samplehtml.c

code:sample.c

# 1 "samplehtml.c"

# 1 "<built-in>"

# 1 "<command-line>"

# 1 "samplehtml.c"

<html>

<head>

<title>ABCについて</title>

</head>

<body>

ABCというシステムについて解説します。

<br>

ABCは、2013年に...

</body>

</html>

m4: general-purpose macro processor

Described in “Software tools”

Using a macro processor for HTML

m4 example (1)

code:m4.txt

% cat sample1.m4

define(name,Masui)

My name is name.

My `name' is name.

m4 example (2)

code:sample2.m4

define(a,100)

define(b,200)

aはbとifelse(a,b,等しい,異なる)

m4 example (2)

code:sampl2.m4

% cat sample2.m4

define(a,100)

define(b,200)

aはbとifelse(a,b,等しい,異なる)

%sample2.m4

100は200と異なる

m4 example(3)

code:sample3.m4

% cat sample3.m4

define(eq,$1は$2とifelse($1,$2,等しい,異なる))

eq(100,100)

eq(100,200)

m4 example (3)

code:sample3.m4

% cat sample3.m4

define(eq,$1は$2とifelse($1,$2,等しい,異なる))

eq(100,100)

eq(100,200)

% m4 sample3.m4

100は100と異なる

100は200と異なる

m4 example (4)

code:sample4.m4

% cat sample4.m4

define(eq2,$1は$2と`ifelse($1,$2,等しい,異なる)')

eq2(100,100)

eq2(100,200)

m4 example (4)

code:sample4.m4

% cat sample4.m4

define(eq2,$1は$2と`ifelse($1,$2,等しい,異なる)')

eq2(100,100)

eq2(100,200)

% m4 sample4.m4

100は100と等しい

100は200と異なる

m4 example (5)

code:sample5.m4

% cat sample5.m4

define(times2,`ifelse($1,$2,$3,

`times2(incr($1),$2,$3)$3')')

define(times,`times2(1,$1,$2)')

times(4,`<br>')

m4 example (5)

code:sample5.m4

% cat sample5.m4

define(times2,`ifelse($1,$2,$3,

`times2(incr($1),$2,$3)$3')')

define(times,`times2(1,$1,$2)')

times(4,`<br>')

% sample5.m4

Exercise: m4

Run 'm4' command on the Unix shell

Use "Terminal.app" on your Mac

Type something, and type enter

enter define(a,1) , and enter a

Markup languages

Specify formatting primitives in the text

TeX, HTML, etc.

Special tags

\bf{text}

TeX example

code:sample1.tex

% cat sample1.tex

Hello, world!

\end

% tex sample1.tex

This is TeX, Version 3.141592 (Web2C 7.5.4)

(./sample1.tex 1 )

Output written on sample1.dvi (1 page, 228 bytes).

Transcript written on sample1.log.

Generated PDF

http://gyazo.com/e2b8b546e8cd83d6fe6d32b71183bc25.png

Example of macro usage

code:sample2.tex

% cat sample2.tex

\def\name{Masui}

Hello, \name!

\end

% tex sample2.tex

This is TeX, Version 3.141592 (Web2C 7.5.4)

(./sample2.tex 1 )

Output written on sample2.dvi (1 page, 224 bytes).

Transcript written on sample2.log.

Generated PDF

http://gyazo.com/abd5ba74f0a1d77593b28dcaf8c71c5b.png

Macro usage example

code:sample3.tex

% cat sample3.tex

This is \TeX.

\par

\centerline{$-b \pm \sqrt{b^2 - 4ac} \over 2a$}

\par

\font\helv=Helvetica scaled \magstep 5 \relax

\helv

This is \TeX.

\end

Generated PDF

http://gyazo.com/a16220b1db81c22585964bf2ace15803.png

LaTeX example

code:sample4.tex

% cat sample4.tex

\documentclass{article}

\begin{document}

\LARGE

abcde

\begin{quotation}

quoted string

\end{quotation}

\end{document}

% latex sample4

This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)

entering extended mode

(./sample4.tex

...

Output written on sample4.dvi (1 page, 292 bytes).

Transcript written on sample4.log.

Generated PDF

http://gyazo.com/c1ddd74ed362b1e18b3e4101313033d1.png

Exercise: TeX

Create sample.tex

code:sample.tex

\documentclass{article}

\begin{document}

\LARGE

abcde

\begin{quotation}

quoted string

\end{quotation}

\end{document}

Run TeX

% tex sample.tex

% dvipdfm sample

% open sample.pdf

Other markup languages

Scribe

roff

RTF

HTML

Scribe

Created by Briand Reid in 1978

Popular at CMU around 1990

Used for papers and technical reports

Almost extinct at this moment

Scribe Manual

http://gyazo.com/6febaa039e718d9d3cf4941c73d7909b.png

Scribe notation

http://gyazo.com/2e9f49c5d0530b4d33576a9b1dfb4e12.png

Scribe notation

http://gyazo.com/038e5a90cac8236632cb23f5cee3f15f.png

Remains of Scribe

BibTeX notation

code:sample.bibtex

@BOOK{Lamport:LaTeX,

AUTHOR = "Leslie Lamport",

TITLE = "LaTeX User's Guide and Document Reference Manual",

PUBLISHER = "Addison-Wesley Publishing Company",

ADDRESS = "Reading, Massachusetts",

YEAR = "1986" }

Roff / Runoff

Text formatting system for Unix

Used for showing manuals

nroff, troff

/usr/share/man/man1/man.1

code:man.1

.SH DESCRIPTION

.B man

formats and displays the on-line manual pages. If you specify

.IR section ,

.B man

only looks in that section of the manual.

.I name

is normally the name of the manual page, which is typically the name

of a command, function, or file.

However, if

.I name

contains a slash

.RB ( / )

then

.B man

interprets it as a file specification, so that you can do

.B "man ./foo.5"

or even

.B "man /cd/foo/bar.1.gz\fR.\fP"

.PP

Trof execution

% groff -man man.1 > man.ps

http://gyazo.com/32cab038afc431a0ea7f65af650f06a2.png

RTF (Rich Text Format)

Markup notation for Microsoft Word

Similar to TeX

Not human-readable

code:sample.rtf

{\rtf1\ansi\ansicpg932\cocoartf949\cocoasubrtf430

{\fonttbl\f0\fswiss\fcharset0 Helvetica;}

{\colortbl;\red255\green255\blue255;}

\paperw11900\paperh16840\margl1440\margr1440\vieww9000\viewh8400\viewkind0

\pard\tx566\tx1133\tx1700\tx2267\tx2834\tx3401\tx3968\tx4535\tx5102

\tx5669\tx6236\tx6803\ql\qnatural\pardirnatural

\f0\i\fs50 \cf0 abc}

HTML

Most widely-used markup language

Based on SGML

METAFONT

Font description language by Prof. Knuth

Describe font by pen movement

Outline definition available

METAFONT example

http://gyazo.com/bdbfdd820ea582d55c427c46209375f8.png

code:sample.metafont

beginchar(65,10pt#,10pt#,0pt#);

pickup pencircle xscaled 1.2pt yscaled 0.5pt rotated 120;

z1=(0.1w,0.75h);

z2-z1=whatever*(8,1);

x2=0.9w;

z3=(0.8w,0.1h);

draw z1--z2{z1-z2}..{z2-z1}z3;

labels(1,2,3);

endchar;

end.

METAFONT status

Almost nobody is using METAFONT

NO Japanese font

Researchers tried to use METAFONT for Japanese characters and failed

History of fonts

Fonts are distributed by font companies

Bitmap fonts

Outline fonts

Combination of bitmap fonts and outline fonts

Antialiasing

ClearType

Literate programming concepts

Integrate program codes and documents

Treat them as a single literature

WEB: a literate programming system by Knuth

Weave: Weave: convert a document to a TeX text

Index, TOC, etc. are automatically generated

Tangle: Tangle: extract program codes from the document

Weave and Tangle

http://gyazo.com/2f70d3aacf06e3f7135587037cd86ea4.png

The TeX Book (1984)

http://gyazo.com/84e7403307e75b2f6487c832dc59e14e.png

Merits of literate programming

Source codes can be written at any place

Codes can be placed in arbitrary place in a document

Documents and codes are always consistent

Programmers feel uneasy when codes and documents are inconistent

Creating a literature is more satisfying than writing a code

Documents related to programming

Source codes

Documents for the algorithm

Documents related to programming

Source codes

Documents for the algorithm

Authors, timestamps, version numbers

Licenses

Testing environments

Build environments

c.f. Can source codes explain everything

All the behaviors of the code is visible

Documents are not required if source codes are self-explanatory

Good practice to write beautiful codes

Problems of code-centric approach

Codes are sometimes difficult to understand

Not appropriate for handling figures, shapes, etc.

What's this?

code:root.rb

v = 3.141592

100.times {

v = (v + 2.0/v) / 2.0

}

puts v

What's this?

code:sample.rb

v = 3.141592

100.times {

v = (v + 2.0/v) / 2.0

}

puts v

Answer: calculating $ \sqrt{2}

What's this?

code:shift.c

int hatena(int b)

{

int n;

n = (b >> 1) & 03333333333;

n = b - n - ((n >> 1) & 03333333333);

n = ((n + (n >> 3)) & 0707070707) % 077;

return n;

}

What's this?

code:what.c

int what(int b)

{

int n;

n = (b >> 1) & 03333333333;

n = b - n - ((n >> 1) & 03333333333);

n = ((n + (n >> 3)) & 0707070707) % 077;

return n;

}

Answer: calculating the number of bits

e.g. what(10) => 2

Programming with figures

State transiton diagrams

c.f. State transition programming

Geometric problem solving

GUI programming

c.f. Visual programming

PrettyPrinting

Printing programs beautifully

Using different fonts and colors

(sample of prettyprinting here)

PrettyPrinting

Beautiful output

Easy to read

Not popular these days

Nobody prints source codes?

People like Courier fonts?

sample.web

code:sample.web

\centerline{\bf Hello World}

\vskip .5cm

This is a very simple example of the WEB system.

@ First, we define a text to be displayed.

@<Content to print@>=

'Hello, Literate Programming!'

@ This is the mainprogram of the Pascal program.

An expression surrounded by $\langle$ and $\rangle$

is substituted by its definition.

Program HelloWorld(output);

begin

@<Do something inside a program@>

end.

@ You can define an identifier after reference in WEB language.

@<Do something inside a program@>=

writeLn( @<Content to print@> )

Run weave

Generating sample.tex

code:sample.tex

\input webmac

\centerline{\bf Hello World}

\vskip .5cm

This is a very simple example of the WEB system.

\M1. First, we define a text to be displayed.

\Y\P$\4\X1:Content to print\X\S$\6

\.{\'Hello,\ Literate\ Programming!\'}\par

\U3.\fi

\M2. This is the mainprogram of the Pascal program.

An expression surrounded by $\langle$ and $\rangle$

is substituted by its definition.

\Y\P$\\{Program}\\{HelloWorld}(\\{output})$;\6

\&{begin} \37\X3:Do something inside a program\X\6

\&{end}.\par

\fi

\M3. You can define an identifier after reference in WEB language.

\Y\P$\4\X3:Do something inside a program\X\S$\6

$\\{writeLn}(\X1:Content to print\X)$\par

\U2.\fi

\inx

\:\\{HelloWorld}, 2.

\:\\{output}, 2.

\:\\{Program}, 2.

\:\\{writeLn}, 3.

\fin

\:\X1:Content to print\X

\U3.

\:\X3:Do something inside a program\X

\U2.

\con

Generated document

http://gyazo.com/eb861108fe4d51892b5f3d665a24be47.png

Running Tangle

sample.p generated

With original line numbers

code:sample.p

{2:}Program HelloWorld(output);

begin{3:}writeLn({1:}'Hello, Literate Programming!'{:1}){:3}end.{:2}

Problems of WEB

Restrictions of TeX

e.g. Difficult to show figures

Restrictions of programming languages

Only TeX+Pascal is supported

C-based WEB available

Difficult to handle libraries

Solutions

Using HTML instead of WEB

Allow various languages

Simple WEB by Masui

Use HTML instead of WEB

wtangle for retrieving source codes

Example of simple WEB

http://gyazo.com/02ab50c897c851f52cda44c61ce311c5.png

Source document of simple WEB

code:test.web

<h3>初期化処理</h3>

<a href="#prologue"><code>prologue()</code></a>では変数の初期化などを行なう。

<code>%SIG<code>に関数名を指定しておくと、

signalが発生したときに<code>$SIG{シグナル名}</code>の関数が呼ばれる。

ここではSIGINTなどを受け取った時に

<a href="#finish"><;code>finish()</code></a>が呼ばれるようにしている。

sub prologue {

require 'cacheout.pl'; # <a name=cacheout>複数ファイル出力ライブラリ</a>

if($#ARGV < 0){

print STDERR "wtangle --- Web Tangle\n";

print STDERR "Usage: wtangle documentfiles\n";

exit 0;

}

$SIG{'INT'} = $SIG{'TERM'} =

$SIG{'QUIT'} = $SIG{'HUP'} = 'finish';

}

</pre>

<h3>メインルーチン</h3>

Simple WEB features

Weave not required

Easy to publish

HTML-Based

Problems of literate programming

Not useful enough

Problems of literate programming

Not useful enough

Problems of literate programming

Not useful enough

Nobody edits documents while debugging

Nobody write documents after debugging

Programmers shoule be highly motibated

Wiki-based literate programming

Programming everywhere

Easily link everything

Supports cooperation

A Web sites can be a literature

Wiki Wiki Web

The first Wiki

300 lines Perl code

The system code is written on the Wiki

Wiki Wiki

http://gyazo.com/d02b8155727323050020b4507e3962dc.png

Text input system for Android

Written on 'Gyazz'

http://gyazo.com/310d6680c8c2dbdff5d86ed33d12eaaf.png

Source code and other text

http://gyazz.com/HelloIME/ http://gyazo.com/79ff4a880b0ca1b3ae2fc92bfb2146ab.png

Programming on Scrapbox

Writing code in Scrapbox pages

Square root

Calculate$ \sqrt{2}

Run the code below

code:sqrt.js

v = 3.141592; // dummy initial value

for(var i=0;i<100;i++){

v = (v + 2.0/v) / 2.0;

}

alert(v);

Random walk

Run the code below

code:random.js

setup = function(){

posx = 200, posy = 200

createCanvas(400, 400)

strokeWeight(0)

}

draw = function(){

fill('red')

rect(0,0,400,400)

posx += random(-2,2)

posy += random(-2,2)

fill('blue')

rect(posx,posy,4,4)

}

Adding random numbers

Run the code below

code:randomadd.js

function setup(){

createCanvas(400,400)

histogram = []

for(i=0;i<100;i++) histogrami = 0

strokeWeight(0)

}

function draw(){

val = 0

for(i=0;i<10;i++) val += random(1/10)

histogramMath.floor(val * 100) += 1

fill('yellow')

rect(0,0,400,400)

fill('blue')

for(i=0;i<100;i++)

rect(i*4,400-histogrami,4,histogrami)

}

Today's topic

Literate programming

Markup languages

Macro programming

Text formatting system

Runninb programs on Scrapbox