Rediscovering BerkeleyDB on the Raspberry Pi

BerkeleyDB (BDB) is an embedded database designed for in-application datastores and small devices. At its most basic, it’s a simple key-value store of byte arrays.

Like sqlite, the entire database is self-contained in a series of files, with no process or network port overhead.

Unlike sqlite, it also has good support for managing concurrent transactions and more advanced database features such as live backups and replication.

Querying is also surprisingly advanced. Beyond hashtable style key lookups, the keys themselves are indexable and searchable, and thanks to the secondary index feature, it’s also possible to do complex queries within the data values themselves.

While it’s possible to get a full-fledged relational dbms like PostgreSQL running on a Raspberry Pi, the fact that BDB runs with minimal resources is useful in a constrained environment, especially if you have disabled swap, and undertaken other cruel and unusual measures to preserve SD card life so that your Pi can run as close to 24x7x365 as possible.

Back in 2009, I was using it extensively on a messaging and workflow project, and I revisted that code when it came time to decide on a datastore for a new, long-running server application designed to run on a Pi.

I’ve cleaned-up that codebase and made it available as a public repo on github: https://github.com/dpapathanasiou/pyBDB

Email as the Ultimate API

Friends have asked me why I designed TeamWork.io and ReadZap.com around text and email, when it seems the current trend is towards visuals and graphics, especially on phones and tablets.

It was something of a lark, in that it combines an obsession with Mailinator’s technical architecture, along with a now-forgotten coordination system theorized by Terry Winograd and Fernando Flores.

But there are more practical reasons for doing it as well.

First of which is: this is how every project starts and evolves.

I.e., someone creates a spreadsheet of things to do, and maintains it (maybe, if they’re really disciplined) for a few weeks at the most.

Then, all the subsequent important information devolves into a mass of random emails among the project participants.

So I thought, why not leverage those two things, and use them to keep an automated record?

And the next reason is more compelling: the email protocol, even as originally designed, is also a nifty application programming interface in its own right.

In a nutshell, email is:

So no dedicated mobile apps are necessary, nor is any special effort required for responsive web design.

The reactions, interestingly enough, have been either:

I love it, so simple!

or

I hate it. It feels like going backwards, to programing in DOS.

Which is good, actually, since the worst thing any project can suffer from is indifference.

Andy Grove’s 100 point guide to being a better manager

I went to see Ben Horowitz give a talk at Columbia a few months ago.

He was there pitching his new book, The Hard Thing About Hard Things: Building a Business When There Are No Easy Answers, which was less entertaining to read than listening to him in person, but it did raise a salient point: the best guide on being a manager was written thirty years ago, by Andy Grove.

Sure enough, High Output Management has stood the test of time.

Grove approaches the problem as an engineer, with a specific, deterministic way of looking at the world, which is a refreshing change from other business management books.

Here’s the worksheet from the book’s epilogue. In Grove’s words, “If you do at least 100 points worth of what you find here, you’ll be a distinctly better manager for it“.

Production

Action Points
Identify the operations in your work most like process, assembly, and test production   10
For a project you are working on, identify the limiting step and map out the flow of work around it   10
Define the proper places for the equivalents of receiving inspection, in-process inspection, and final inspection in your work. Decide whether these inspections should be monitoring steps or gate-like1. Identify the conditions under which you can relax things and move to a variable inspection theme.   10
Identify half a dozen new indicators for your group’s output. They should measure both the quantity and quality of the output.   10
Install these new indicators as a routine in your work area, and establish their regular review in your staff meetings.   20
What is the most important strategy (plan of action) you are pursuing now? Describe the environmental demand that prompted it and your current status or momentum. Is your strategy likely to result in a satisfactory state of affairs for you or your organization if successfully implemented?   20

Leverage

Action Points
Conduct work simplification on your most tedious, time-consuming task. Eliminate at least 30 percent of the total number of steps involved.   10
Define your output: What are the output elements of the organization you manage and the organizations you can influence? List them in order of importance.   10
Analyze your information- and knowledge-gathering system. Is it properly balanced among “headlines,” newspaper articles,” and “weekly news magazines”?2 Is redundancy built in?   10
Take a “tour”. Afterward, list the transactions you got involved in during its course.   10
Create a once-a-month “excuse” for a tour.   10
Describe how you will monitor the next project you delegate to a subordinate. What will you look for? How? How frequently?   10
Generate an inventory of projects on which you can work at discretionary times.   10
Hold a scheduled one-on-one3 with each of your subordinates. (Explain to them in advance what a one-on-one is about. Have them prepare for it.)   20
Look at your calendar for the last week. Classify your activities as low-/medium-/high-leverage. Generate a plan of action to do more of the high-leverage category. (What activities will you reduce?)   10
Forecast the demand on your time for the next week. What portion of your time is likely to be spent in meetings? Which of these are process-oriented4 meetings? Mission-oriented5 meetings? If the latter are over 25 percent of your your total time, what should you do to reduce them?   10
Define the three most important objectives for your organization for the next three months. Support them with key results.   20
Have your subordinates do the same for themselves, after a thorough discussion of the set generated above.   20
Generate an inventory of pending decisions you are responsible for. Take three and structure the decision-making process for them, using the six question approach6.   10

Performance

Action Points
Evaluate your own motivational state in terms of the Maslow hierarchy. Do the same for each of your subordinates.   10
Give your subordinates a racetrack: define a set of performance indicators for each.   20
List the various forms of task-relevant feedback your subordinates receive. How well can they gauge their progress through them?   10
Classify the task-relevant maturity of each of your subordinates as lowe, medium, or high. Evaluate the management style that would be most appropriate for each. Compare what your own style is with what it should be.   10
Evaluate the last performance review you received and also the last set of reviews you gave to your subordinates as a means of delivering task-relevant feedback. How well did the reviews do to improve performance? What was the nature of the communication process during the delivery of each?   20
Redo one of these reviews as it should have been done.   10

1 Hold output at the “gate” and check it before allowing it to pass to the next stage of the production process.
2 This book was published in 1983
3 Whereby the subordinate explains what is going on, and what is bothering him (i.e., it is the subordinate’s meeting)
4 Where knowledge is shared and information is exchanged; held on a regular basis
5 Designed to produce a specific decision; usually held on an ad-hoc basis
6 What will the environment demand from you, your business, or your organization?
What are you producing now?
what more (or less) do you need to do to reconcile the gap?
Where do I want to go (i.e., what is the objective?)
How will I pace myself to see if I am getting these?
What is the period of time to focus on the objectives? (focus is important, to know what to say “yes” to, and “no” to)

Debian 7 Server + Fluxbox via VirtualBox

This is a link to a clean debian 7 server image as a virtualbox vdi package.

I’ve been using it quite a bit as the basis for different clones of sites and web services I’m refactoring, and it also came in handy cleaning up the Heartbleed mess.

It was built essentially the same way as described here in an earlier post, with Guest Additions installed, and just a small set of core development packages:

# apt-get install dkms build-essential linux-headers-$(uname -r)
# apt-get install xorg fluxbox
# apt-get install roxterm git mercurial python-all-dev python-pip python-lxml python-magic python-imaging

[after "Inserting" the Guest Additions CD via the Devices menu]
# mount /dev/dvd /media/cdrom
# cd /media/cdrom
# sh ./VBoxLinuxAdditions.run

As before, I’m using fluxbox as the window manager, though that can be changed or removed as needed.

To install it in your environment:

  1. Download the Debian7BaseServer.tar.bz2 file and unpack the archive
    (use tar jxf Debian7BaseServer.tar.bz2 from a terminal on both Linux and Mac OSX)
  2. Start virtualbox and create a New Virtual Machine, using the uncompressed vdi file as the hard drive

The password for both root and the sole regular account — averagejoe — is all4one but that can (and should) be changed by using the passwd command after first login.

To bring up the window manager as either user, type startx after logging in successfully.

 

Zen Thought of the Day – 31st

31th

大望を持て
全ては可能だ

With ambition
Everything is possible


Ongoing: Now that the series is over, I’ve turned all these posts into a daily aphorism terminal app for Mac OSX and Linux, modeled after the old unix fortune application.

Clone it from github here: https://github.com/dpapathanasiou/zen-thought

Zen Thought of the Day – 30th

30th

自分の心が映ってる
自分の周囲の現象に

The image of the people around me
Is reflected onto my heart

Zen Thought of the Day – 29th

29th

何はとも
あれ周囲に笑顔で
話は出来る人が居る境遇に
感謝

Anyway
To the environment where people can
Talk with big smiles on their faces
Thank you

Zen Thought of the Day – 28th

28th

いえいえあなたが
いてくれたお陰です
本当にありがとう

No, no I’m very indebted to you
Because of you, I am here
Thank you very much

Zen Thought of the Day – 27th

27th

ありがとう
あなたに出逢えて良かった

Thank you
Because of you, I am here

Zen Thought of the Day – 26th

26th

奪えば奪われ
与えれば与えれる

If I take Things will be taken from me
If I give I can receive