Notes from TUG2008 in Cork: Day 4
2008-07-24
These are rough notes from a public event, and any errors or stupidity should be attributed to me and my poor note taking; I hope these notes are useful despite their obvious flaws, and everything should be double checked :)
LuaTeX Images
Hartmut Henkel
[09:50 damn overslept again]
Q: Type3 fonts can’t work well in PDF and this can make PDF submissions of theses rejected…
LuaTeX
Hans Hagen
Why are there so many kind of a program? Well there are lots of people, and there is no need for everyone to be the same.
We have heard about LuaTeX the program; what are the consequences for macro packages? Where does TeX end and Lua start?
We see no reason to really change TeX, it works and people are happy with it. We don’t provide solutions, we provide the way to provide solutions. There are no common views on how pending issues can be solved.
We open up the internals of TeX for the macro programmers. ConTeXt mkIV shows this off. ConTeXt started off as calling Lua and piping its output to TeX - a caller primitive and a print function.
Then we opened up registers, to access counters and box dimensions. Then we replaced file handling, normally done by kpse library; reading zip files, HTTP or FTP, and so on became possible. TeX itself is unaware of all that, and we moved multipass data to Lua.
Then input encodings were done in Lua, supporting UTF-16 even, and this was the first time the TeX part became smaller and the Lua part becoming larger.
More complex Lua scripting was added, for MetaPost conversion to pipe data to La and print back PDF literals.
Then we got access to node lists, so we could manipulate glyph related data, for robust case swapping and such.
Then we used Lua for font loading, including preparing for OpenType processing. Reading from AFM replaces TFM, for great Type1 support, and ‘font encoding’ has been removed from ConTeXt mkIV now and only maths still needs TFM files - but that will go soon too.
The Oriental TeX project, we started writing support for advanced OpenType features; everything is Lua controlled, and Lua is so fast that major node-crunching is okay. Zapfino can have 1,000,000s of node operations with the complex layout of that font. This is great for a developer as you can afford to implement things that waste time. Runtime virtual font building can be used to construct missing glyphs, and features can be supported dynamically. This kind of stuff is really new in TeX.
The generic attribute mechanism was redone. This was used to specify fonts, but when redone can be used for other things - no big speedup, but more robust and flexible.
XML-FO has a good vertical spacing model.
We really need the TeX-Gyre math fonts!
The stream driven MkII xmlhandling was replaced by a tree based mechanism for arbitrary access/flushing/manipulation of XML nodes, so you can say ‘give me all of these tags with that attribute value’ - MathML support was done as a use case example. Its very fast.
There is a new img library that is powerful.
The sectioning, numbering and lists is done in Lua, apart from the typesetting part; more data is carried around and retained, and its hard to keep it compatible, but possible.
And there are some minor parts of the typesetting operations being done by Lua, but mostly that is left to TeX that does it well.
Authors can just use Lua, forget about the TeX internals and just pass out and return data from/to TeX.
Macro authors, can get information from TeX, use it for your own calculations, and add the results to TeX; or even manipulate that data with Lua and replace the original TeX dataset, and you can replace components of TeX (like file handling) with Lua code to be more flexible and modern, and you can make use of OpenType features, and even replace typesetting parts like hyphenation, kerning, paragraph building - and add new features not seen before :)
…
Macro writing will become more powerful and convenient.
LaTeX Announcement
Frank Mittelbach
The LaTeX source have been in SVN for a while; yesterday they went public, so development now happens live online, so if there are bug fixes, people can get them quickly, and see how things develop. http://www.latex-projecr.org/svnroot/ or something, there will be a post on the website.
And the last 2 years we’ve been saying we’ll move from TeX to another engine, and we are going to e-TeX officially now.
Don’s Punk Font
Hans Hagen
11:05
How it started? We had a mechanism for making virtual fonts, and long ago I ran unto a font by DEK, “punk,” and it was a METAFONT bitmap font. Nowadays no one uses bitmap fonts, and pdfTeX can pretend to make bitmaps fonts using glyph containers. When MPLib started, I tried to give that a go. Taco converted the METAFONT file into something more MetaPost like, and I wrote a virtual font for that.
How does it work? The MetaPost file is processed using MFPLAIN format, and if you look carefully you can see there are differences. There are 10 in fact, for various sizes. These pictures are converted to a PDF stream and stored in the mkIV font cache. It looks like a simple font, but its quite complex actually. At run-time a font is assembled from these pictures, and we can add missing glyphs composed like diacritic ones. We use an attribute signal to say some text has to be punked, and one of the node parsers picks up this signal and randomly chooses a font. The shapes end up in the stream as in-line PDF code as a result of the virtual font. But this means its not search-able.
Virtual fonts have a great potential. Not a real reason to start using LuaTeX, but shows the potential. The type options of LuaTeX are rather minimal (not there) and we need proper PDF text stream support to make it work.
The font need some subtle fine-tuning, and more character support; symbols and math support for example ;) and perhaps we could ask Herman Zapf to do Punk Nova.
And the MetaPost library needs to be made suitable for making fonts, kerning and such. LuaTeX needs to be extended for proper Type 3 handling.
In the future Taco will provide MetaPost features for outputting char strings, so you can have run time glyph generation in LuaTeX.
Eventually, we can apply this mechanism to sophisticated ‘hand writing’ like fonts.
The font itself is punkfont.mf and is available now
Here is a quick walk through of the code.
LuaTeX’s impact on the TeX world: A Lua file, and a TeX file, that does this never before possible. This is an example of this.
Q: how to search?
A: you can use “invisible” text to duplicate the content that makes it search-able.
Languages for Bibliography Styles
Jean-Michel Hufflen
11:25
Many programming languages in computer science are specialised, and they relate to the software qualities. “Object Orientated Software Construction” in 1997 by Mertrand Meyer is a book about quality of software. There are 2 kinds of qualities; externals (seen by users) like correctness to perform its tasks, robustness in abnormal conditions, extendability for some changes and re usability in other programs. Efficiency, portability, and usability too.
Then internal quality is seem only by developers; readability, modularity. Modularity is decomposibility, and composibility, can parts be combined with each other? Can you understand a part of the program without knowing the rest? And continuity, can a small change in the specification result in a small change in the program?
The task of a bibliography process er is to walk a bibliographic database for ‘keys’ and sort them into an order, and arrange each reference in a bibliographic style - which may be language dependent.
From a user perspective, you want to design and implement your own bibliographic styles, and perhaps add new features to the processor to do this.
A bibliographic processor ought to work with LaTeX and ConTeXt and XeTeX and LuaTeX; and output HTML and RTF also.
BibTeX is the LaTeX bibliographic processor, using the BST language for bibliographic style. It uses a stack model and a postfix notation.
It has good qualities: correctness - I never crashed it! - and robust - a big file with many syntax errors is no problem - but its extendability is limited, and its re-usability is too. And extending BibTeX is tedious for programming-like things. Its modularity is poor; some functions can be changed but need stack programming stuff, popping and pushing things on the stack. Continuity is average; changes are easy especially for layout, but other kinds of changes are tedious.
Can BibTeX be used for other word processes than LaTeX? Theoretically yes, but practically it is hard as many users put LaTeX commands inside field values. And some notations are hard wired like “~” for unbreaking spaces that is important for authors’ names.
BibTeX’s output is marked up with commands and these interface with citation and formatting functions in TeX; natbib, jurabib, biblatex, etc. BibTeX is used to search databases and sort references in that case.
Another approach is to use XML like formats; .bib files can be converted to a XML format and that is processed by XSLT. bib2xml is such a tool. Most accent commands are expanded to the Unicode characters.
mlBibTeX uses an XSLT for bibliography styles, and this introduces inheritance about natural languages’ specification; it uses Scheme to do some programmatic things. It has a compatibility mode for .BST styles, and its usable if you know Scheme, although it needs better integration.
DSSSL is an old style specification language that uses s-expressions, and was used in SGML. The “Core Expression Language” is a subset of scheme, and it has the full power of a programming language.
Another choice is Perl as a bibliography processor. Bibulus is such a tool. Compact, modular, extensible, efficient and good for HTML especially. But multilingual features are limited; the homepage says “only for developers.”
Tib is old, for Plain TeX, but easy to use.
BibTeX++ compiles BSD into Java classes, but this has similar limitations to BibTeX.
Common Lisp too - cl-bibtex allows the use of Common Lisp functions, and that’s great if you are a Common Lisp hacker ;)
TeX is a great typesetting engine, but not a general purpose programming language, and so delegating that part is good - LuaTeX does this very well.
I think the XML approach of mlBibTeX is promising and adds something these alternatives don’t have. I think the Lua/TeX balance is similar to mlBibTeX’s Scheme/XSLT balance.
Observations of a TeXnician for hire
Boris Veytsman
11:55
A story in 6 lessons.
For those old timers, perhaps my comments are debatable, but here are some personal observations, take what you can.
- “The usefulness of audacity”
My work as a consultant started years ago; I looked at the rates in TUGBoat, and its $35 a year to be listed as a consultant there. So for $35 I thought I’d be a consultant. I had read some books and used LaTeX but thought I’d give it a try. Nothing happened. So I wrote to Karl Berry and asked if there was anything I could do to help. He said, why don’t I help translate documentation into Russian. I said okay, and started this large and unpaid work. After that work, I got a call from a scientific society, accepting submissions for publications in TeX, and they needed someone to teach the support team TeX. I had a day job as an engineer, and a night job as a professor, but I love teaching, and so I said okay. I asked how the heard about me, and they said Karl Berry recommended me.
2 “Volunteering helps to get paid”
With word of mouth, I got more customers, and eventually I earned a lot of money, not as much as Karl ;) but some.
Then I had to pay taxes. But I can claim business expenses against those taxes, and in 2006 there was a PracTeX conference. A few hours drive away, quite expensive to drive there, I met a lot of nice people who I respect only via email and who I respect a lot. And I got some work from the people I met there.
- “Going to conferences helps to get paid”
We have a niche market: most of my work has been with publishers, and its boring work, but they give you specs and you write a style file and its simple clean work. Clients expect you to be an expert in TeX, friends of TeX and friends of friends of TeX. You can’t tell them “I don’t know about that”. This is the lesson
- “The way of the Omnivore”
But also,
- “You cannot do everything”
Typography is both an art and a science; you need the compositors, opinionated people, they are the boss, and you can have your opinion but only tell it when it is asked for. A few times people have commended me on my deep knowledge of typography. My secret is that, whatever they say, I will do.
- “Why This is fun”
Firstly, I was always learning. I’ve learned about Hebrew typography, about PostScript and PDF. It was fun to learn, I’d like to learn it even if I wasn’t being paid. TeX is totally different to my other work in engineering and teaching, so its a kind of distraction for me.
If you do engineering, most companies have their own opinions about secrecy, doing things like patents even on software ideas, and its bad for your karma. But if you are a publisher, they want things to be published and available as free software - its good for my well being and enjoying what I am doing.
Some practical advice:
I do work and teaching, but consulting is different work. The most important part is:
- Communication
You must listen to what they want, and make them happy. Publishers have editors, authors, compositors, and you have to make them all happy. The main part of the work is communicating to everyone, and to do that, you must understand their emotions. The typical emotion of a TeX user?
Fear.
TeX is a big intimidating system, they are afraid it will break in the middle of their production and they will have no way out. So the next important thing is:
- Support
You can’t say, “here is my system, its $50 a question” - they will be afraid and you don’t want that.
The third thing? Suppose a user comes to you with a problem, and you expected it and on page 53 you explained the problem and the solution. They already paid for the system and the manual.
I see this as my problem, as an author. I didn’t explain it well, or fun to read so they didn’t read it well, and so I will always apologise, say I will rewrite the manual and explain the solution now too. So the third most important thing with TeX consulting:
- Documentation
Talking, explaining, explaining again. That’s what you are doing. So when doing the documentation, you remember that the support is free. So the documentation will save you money by having less support.
In conclusion:
Do what you enjoy doing and be generous. At some point somebody
will pay you!
Don’t chase money for every single thing you do.
If you are a consultant, I hope this will be useful for you; if you are not, you have my name… ;)
If you ask me a question, and I can’t find the answer in 15 minutes, its free. If its such an interesting question I want to know the answer, its also free.
Q: Have you worked for the government?
A: Maryland has.
Q: If you work for the government, it can take a long time for them to pay you ;)
A: Perhaps - I found them to be one of my best customers in terms of prompt payment :-)
12:15
What are the upcoming TeX conferences?
EuroTeX 2009, in Delft, August 31 - Sept 4. This will also be the 3rd ConTeXt meeting.
TUG2009 will be at University of Notre Dame, Southbend, Indiana (100 miles west of Chicago, on an Amtrack route, and on a major interstate highway 80, and on the Internet too - during the conference - and hope to see you there. Its July 28th-31st.
The Notes from TUG2008 in Cork: Day 4 by David Crossland, except the quotations and unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.
Comments
Leave a Reply