FANDOM


These instructions to install the Fosm API server were developed for Linux Mint 18, but should work on modern Debian/Ubuntu systems. Some tweaks would be required for other distros and systems. You need admin privileges to perform several of the steps. It is recommended you have plenty of hard drive space: the empty database takes about 300Gb. The populated fosm database is around 700Gb (as of 2016).

Using a web browser, download the GT.M database from: https://sourceforge.net/projects/fis-gtm/files/GT.M-amd64-Linux/V5.5-000/ and get the file: gtm_V55000_linux_x8664_pro.tar.gz

This guide explains how to install the system into the /srv/fosm path, but this can be changed if needed. The software will be installed under the www-data username and group, because it allows editing by multiple authorized users and the server is largely web based anyway. This guide uses nano as an editor, but any other will do just fine, such as xed, pluma, gedit, emacs, vi, etc. Add yourself to the www-data group:

sudo nano /etc/group

Scroll down. Add your linux username to the end of the line that is like:

www-data:x:33:

Save the file (in nano, do ctrl-o, enter, then ctrl-x), log out and log back in. Check you are in the www-data group by the command:

groups

Extract and install the GT.M database software:

cd ~/Downloads/
mkdir gtm
tar -xf gtm_V55000_linux_x8664_pro.tar.gz -C gtm
cd gtm
sudo mkdir /srv/fosm
sudo ./gtminstall --installdir /srv/fosm/gtm55 --user www-data --group www-data
cd ..
sudo rm -R gtm

Now we start creating the folder structures needed for the server. (Note: the next section, gtmx does not seem to be necessary, just create an empty folder by "mkdir gtmx" instead.)

cd /srv/fosm
sudo chown www-data:www-data .
sudo chmod ug+rwx .
mkdir planet
mkdir planet/minute-replicate/
mkdir data
mkdir serenji
git clone https://github.com/fosm/fosm-scripts.git scripts
git clone https://github.com/fosm/gtmx.git
git clone https://github.com/fosm/fosm.org-webpage.git
ln -s gtm55 gtm
ln -s gtmx gtm55x
ln -s ../scripts/xapi.gld data/xapi.gld
ln -s fosm.org-webpage/fosm.org www
mkdir /scripts/o
sudo chown www-data:www-data -R .
sudo chmod ug+rwx -R .

If you are installing to a custom path, check the first line of <src>/scripts/profile and set the path you are using.

GT.M requires several environment variables to operate correctly. Assuming you are using bash, add the following lines to the end of your ~/.bashrc file:

source /srv/fosm/scripts/profile
export EDITOR=/bin/nano

Configure the database to use 8k database blocks instead of the default 4k blocks: edit /etc/sysctl.conf (using sudo because it is protected) and add the following lines: (NOTE: is this necessary? Is a reboot needed to apply this?)

# XAPI settings for 8k database blocks
kernel.shmmax = 67108864
kernel.shmall = 67108864

Note: the fosm.org server uses kernel.shmmax=4294967295 ... why?

You should be able to initialize the database at this stage. GT.M should always be run from the folder containing the database i.e. /srv/fosm/data

cd /srv/fosm/data
mupip create

The system might churn the hard drive at this stage to pre-allocate database space.

You can then import a planet dump file into the database. The planet file must use single quotes for xml tags, not double quotes. It also probably requires each tag to be on a separate line (TBC). (NOTE: This process doesn't work! However, it is possible to skip this and continue with a blank slate database.)

cd scripts
mumps loadPlanet.m
cd ../data
mumps -run ^loadPlanet
cat ../hereford.osm > planet.pipe

The apache web server is used to process HTTP requests from the users browser. It also passes API requests to the GT.M scripts. These instructions assume apache 2.4. (fosm.org is currently on version 2.2.9)

sudo apt install apache2
cd /etc/apache2/mods-enabled
sudo ln -s ../mods-available/cgi.load
sudo ln -s ../mods-available/rewrite.load
sudo rm /etc/apache2/sites-enabled/000-default.conf
cd /etc/apache2/sites-enabled/
sudo ln -s /srv/fosm/scripts/fosm.conf fosm.conf

Edit fosm.conf and set a unique password for SERVER_LINK_PASSPHRASE. This password is used by apache to access the private GT.M backend service. If you wish to run the server on a custom port, both fosm.conf and /etc/apache2/ports.conf must be changed.

Restart apache:

sudo service apache2 restart

Use your web browser and look at http://localhost/ to check apache works. If that fails, start by checking:

tail /var/log/apache2/error.log

The website needs more configuration to talk to GT.M. We also need to prepare the xapid service.

cd /srv/fosm/www
rm serverLink.cgi
ln -s /srv/fosm/scripts/serverLink.cgi serverLink.cgi
ln -s /srv/fosm/planet/ planet
cd /srv/fosm/www/api/0.6
rm serverLink.cgi
ln -s /srv/fosm/scripts/serverLink.cgi serverLink.cgi
cd /srv/fosm/www/api/0.6/user/
rm serverLink.cgi 
ln -s /srv/fosm/scripts/serverLink.cgi serverLink.cgi
cd /srv/fosm/scripts
make
cd /srv/fosm/data

We also need to set some variables within GT.M. Run:

gtm

Then issue the following commands (note: the specific values are probably absurd but they more or less work):

s ^osmPlanet("instance")="xapi"
s ^osmPlanet("date")=20140101
s ^osmPlanet("osmSession")="bbb2559f0e425315329ca1f010197858"
s ^serverLink("REST")="rest"
s ^serverLink("port")=6530
s ^serverLink("logLevel")=0
s ^date("timezoneOffset")=8
s ^munin("apiCalls")=27597139
s ^munin("responseDB")=84932980
s ^munin("responseIO")=10466660
s ^munin("responseTotal")=141848508
s ^id("changeset")=1000074436
s ^id("node")=1000594558245
s ^id("pendingUid")=100000383
s ^id("relation")=1000000153460
s ^id("uid")=100000156
s ^id("way")=1000002478076

The server password also needs to be set the same as SERVER_LINK_PASSPHRASE in the apache configuration.

d setup^serverLink

Enter a password, enter any server description string, then press enter when it asks for a second password.  You can check the current values of these parameters with commands like:

zwr ^osmPlanet
zwr ^serverLink
WRITE ^serverLink("REST")

Exit GT.M by typing "halt" or "h" and pressing enter (or ctrl-d).

Start the xapi services with the commands:

cd /srv/fosm/data
export GTMCI=/srv/fosm/scripts/xapi.ci
../scripts/xapid

This should just run without returning to the command line. Just leave this run. Other commands can be performed in a different terminal window.

You can also run the script fosm_up in the scripts directory. This will return to the command line.

cd /srv/fosm/scripts
./fosm_up

You can confirm the service is running using:

netcat 127.0.0.1 6530

If it returns immediately to the command line, the service is not running. Just press enter for netcat and the TCP connection is terminated (by the GT.M server, I think). Next, try doing a basic map query:

http://localhost/api/0.6/map?bbox=-2.742031,52.0509012,-2.7326135,52.0567448

If you have permission problems, try:

cd /srv/fosm
sudo chown www-data:www-data -R .
sudo chmod ug+rwx -R .

Look at:

tail /var/log/apache2/error.log

If you still can't connect to GT.M

cd /srv/fosm/scripts
export SERVER_LINK_IP=127.0.0.1
export SERVER_LINK_PORT=6530
export SERVER_LINK_PASSPHRASE=your-password-here
./serverLink.cgi

That should produce no errors. Incidentally, API errors are logged into the database and can be retrieved by the GT.M command:

zwr ^error

API calls and responses are logged here:

zwr ^log

CrontabEdit

Set cron to run scripts automatically:

sudo crontab -u www-data -e

to contain:

#* * * * * /fosm/scripts/loadDiff
* * * * * cd /fosm/scripts; source exportDiff
#50 * * * * /fosm/scripts/purgeJournal
# Run the planet dump monthly on 1st of month at 5am
#00 05 1 * * /fosm/scripts/planetDump

Create a user accountEdit

We can create a user account but bear in mind email sending is probably not configured. You cannot directly use the email confirmation link. However, we can work around this. Register a new user with a web browser:

http://localhost/user/new

Then on the command line:

cd /srv/fosm/data
gtm

Then issue the command:

zwr ^pendingUser

Look for the "emailToken" parameter. Use a web browser and go to the URL:

http://localhost/user/confirm/emailToken

with the "emailToken" replaced by the actual value. That should enable the user for editing.

You can now start JOSM, change the server URL to:

http://localhost/api

and start editing!

Ad blocker interference detected!


Wikia is a free-to-use site that makes money from advertising. We have a modified experience for viewers using ad blockers

Wikia is not accessible if you’ve made further modifications. Remove the custom ad blocker rule(s) and the page will load as expected.