diff options
author | Ilya Zverev <zverik@textual.ru> | 2018-04-17 19:24:14 +0300 |
---|---|---|
committer | Roman Kuznetsov <r.kuznetsow@gmail.com> | 2018-04-18 11:36:37 +0300 |
commit | 4a3b58ed33aa1f1e1614d0b7d74ffed22bfc8bff (patch) | |
tree | 4a462c05f9db6f9d54fe000aab54d103800c5567 /docs | |
parent | ae680b274aae224fc9c1c7617599e7769c18e227 (diff) |
[doc] Update building instructions
Diffstat (limited to 'docs')
-rw-r--r-- | docs/INSTALL.md | 119 | ||||
-rw-r--r-- | docs/MAPS.md | 39 |
2 files changed, 52 insertions, 106 deletions
diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 3117f65ccb..938a31ff59 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -5,18 +5,20 @@ First, do not forget to initialize a cloned repository, see ## Desktop -You would need Clang, Boost and Qt 5. With that, just run `omim/tools/unix/build_omim.sh`. -It will build both debug and release versions to `omim/../omim-build-<target>`, as -well as OSRM backend for generating maps. Command-line switches are: +You would need Cmake, Boost and Qt 5. With that, just run `omim/tools/unix/build_omim.sh`. +It will build both debug and release versions to `omim/../omim-build-<target>`. +Command-line switches are: * `-r` to build a release version * `-d` to build a debug version -* `-o` to build OSRM backend * `-c` to delete target directories before building +* `-s` to not build a desktop app, when you don't have desktop Qt libraries. +* `-p` with a path to where the binaries will be built. -To build a generator tool only, set `CONFIG=gtool` variable. To skip building tests, -use `CONFIG=no-tests`. If you have Qt installed in an unusual directory, use -`QMAKE` variable. +After switches, you can specify a target (everything by default). For example, +to build a generator tool only, use `generator_tool`. If you have Qt installed +in an unusual directory, use `QMAKE` variable. You can skip building tests +with `CMAKE_CONFIG=-DSKIP_TESTS` variable. When using a lot of maps, increase open files limit, which is only 256 on Mac OS X. Use `ulimit -n 2000`, put it into `~/.bash_profile` to apply it to all new sessions. @@ -29,30 +31,19 @@ and run The `build_omim.sh` script basically runs these commands: - qmake omim.pro -spec linux-clang-libc++ CONFIG+=debug - make -j <number_of_processes> + cmake <path_to_omim> -DCMAKE_BUILD_TYPE={Debug|Release} + make [<target>] -j <number_of_processes> -It will compile binaries to the `out` subdirectory of the current directory. -You might need to export `BOOST_INCLUDEDIR` variable with a path to Boost's -`include` directory. +It will compile binaries to the current directory. For a compiler, Clang +is preferred, but GCC 6+ would also work. -To build the OSRM backend, create `omim/3party/osrm/osrm-backend/build` -directory, and from within it, run: +### Ubuntu 16.04 - cmake -DBOOST_ROOT=<where_is_your_Boost> .. - make +Install Qt 5, CMake and Clang: -### Ubuntu 14.04 - -Install Qt 5.5: - - sudo add-apt-repository ppa:beineri/opt-qt551-trusty sudo apt-get update - sudo apt-get install qt55base - -Set up the Qt 5.5 environment: - - source /opt/qt55/bin/qt55-env.sh + sudo apt-get install qtbase5-dev cmake + sudo apt-get install clang libc++-dev libboost-iostreams-dev libglu1-mesa-dev Do a git clone: @@ -60,33 +51,17 @@ Do a git clone: cd omim echo | ./configure.sh -On Ubuntu 14.04, you'll need a PPA with an up-to-date version of libc++. -You shouldn't need this on newer versions. - - sudo add-apt-repository ppa:jhe/llvm-toolchain - Then: - sudo apt-get install clang-3.6 libc++-dev libboost-iostreams-dev libglu1-mesa-dev - sudo ln -s /usr/lib/llvm-3.6/bin/clang /usr/bin/clang - sudo ln -s /usr/lib/llvm-3.6/bin/clang++ /usr/bin/clang++ - tools/unix/build_omim.sh - -Prepend with `CONFIG=gtool` if only generator_tool is needed. You would need 1.5 GB of memory -to compile `stats` module. + tools/unix/build_omim.sh -r -The generated binaries appear in `omim-build-<flavour>/out/<flavour>/`. -Run tests from this directory with `../../../omim/tools/unix/run_tests.sh`. +Append `generator_tool` if only generator_tool is needed, or `desktop` if you need a desktop app. +You would need 1.5 GB of memory to compile the `stats` module. -To build and run OSRM binaries: +The generated binaries appear in `omim-build-release`. +Run tests from this directory with `omim/tools/unix/run_tests.sh`. - sudo apt-get install libtbb2 libluabind0.9.1 liblua50 libstxxl1 - sudo apt-get install libtbb-dev libluabind-dev libstxxl-dev libosmpbf-dev libprotobuf-dev - sudo apt-get install libboost-thread-dev libboost-system-dev libboost-program-options-dev - sudo apt-get install libboost-filesystem-dev libboost-date-time-dev - tools/unix/build_omim.sh -o - -### Fedora 23 +### Fedora 27 Install dependencies: @@ -96,13 +71,6 @@ Then do a git clone, run `configure.sh` and compile with linux-clang spec: SPEC=linux-clang tools/unix/build_omim.sh -r -### Debian Jessie - -Example [Dockerfile](debian/Dockerfile). In instruction have been compiled Generator Tool and, for routing indices, OSRM backend. I used this command: - - CONFIG=gtool omim/tools/unix/build_omim.sh -cro - - ### Windows We haven't compiled MAPS.ME on Windows in a long time, though it is possible. It is likely @@ -114,15 +82,14 @@ See also [Android compilation instructions](android_toolchain_windows.txt) (also To browse maps in an application, you need first to download some. We maintain an archive of all released versions of data at [direct.mapswithme.com](http://direct.mapswithme.com/direct/). -Place `.mwm` and `.mwm.routing` files to `~/Library/Application Support/MapsWithMe` for -a desktop version. Alternatively, you can put these into `omim/data`, but there -should be a soft link in a build directory: `build_omim.sh` creates it. +Place `.mwm` files to `~/Library/Application Support/MapsWithMe` for +a desktop version. For an Android application, place maps into `/MapsWithMe` directory on a device. For iOS devices, use iTunes. `World.mwm` and `WorldCoasts.mwm` are low-zoom overview maps and should be placed -into a resource directory, e.g. `/Applications/MAPS.ME/Content/Resources` on Mac OS X. +into a resource directory, e.g. `/Applications/MAPS.ME/Content/Resources` on macOS. Placing these into a maps directory should also work. For instructions on making your own maps, see [MAPS.md](MAPS.md). @@ -132,44 +99,34 @@ For instructions on making your own maps, see [MAPS.md](MAPS.md). The generator tool is build together with the desktop app, but you can choose to skip other modules. Use this line: - CONFIG=gtool omim/tools/unix/build_omim.sh -ro - -It is the preferable way to build a generator tool, for it can also build an OSRM -backend (`-o` option). + omim/tools/unix/build_omim.sh -sr generator_tool -Dependencies for generator tool and OSRM backend: +Dependencies for generator tool: * boost-iostreams * glu1-mesa -* tbb -* luabind -* stxxl -* osmpbf * protobuf -* lua ## Designer Tool -The designer tool resides in a branch `map-style-editor-new`. You would need -to check it out before following these steps. - -### Building data +### Building the tool -* Run `CONFIG="gtool map_designer" omim/tools/unix/build_omim.sh`. -* Generate data as usual (either with `generate_mwm.sh` or `generate_planet.sh`). -* For MAPS.ME employees, publish planet data to http://designer.mapswithme.com/mac/DATA_VERSION -(via a ticket to admins, from `mapsme4:/opt/mapsme/designers`). +The designer tool has been merged into the master branch. You can build a package for macOS with: -### Building the tool + omim/tools/unix/build_omim.sh -rt -* Run `omim/tools/unix/build_designer.sh DATA_VER APP_VER`, where `DATA_VER` is the -latest data version published to `designer.mapswithme.com` (e.g. `150912`), and -`APP_VER` is an application version, e.g. `1.2.3`. * If you got "hdiutil -5341" error, you would need to build a dmg package yourself: find a line with `hdiutil` in the script, copy it to a console, and if needed, increase `-size` argument value. * The resulting dmg package will be put into `omim/out`. +### Building data + +* Build both the generator_tool and the designer tool +* Generate data as usual (either with `generate_mwm.sh` or `generate_planet.sh`). +* For MAPS.ME employees, publish planet data to http://designer.mapswithme.com/mac/DATA_VERSION +(via a ticket to admins, from `mapsme4:/opt/mapsme/designers`). + ## Android * Install [Android SDK](https://developer.android.com/sdk/index.html) and diff --git a/docs/MAPS.md b/docs/MAPS.md index 2c8ffcf53f..195d87843e 100644 --- a/docs/MAPS.md +++ b/docs/MAPS.md @@ -10,7 +10,7 @@ It also can include a pre-calculated routing index for car routing. We build map For building maps, you need compiled Generator Tool and, for routing indices, OSRM backend. See [INSTALL.md](INSTALL.md) for compilation instructions. We usually use this line: - CONFIG=gtool omim/tools/unix/build_omim.sh -cro + omim/tools/unix/build_omim.sh -crs Scripts described here require OSM C Tools, which are maintained as a submodule of the omim repository. @@ -53,11 +53,11 @@ This is a shortcut for following options: variable if it's not in `$HOME/planet/planet-latest.o5m`). * `-l`: filter and process coastlines, creating `WorldCoasts.geom` and `.rawgeom` files. * `-w`: generate overview maps, `World.mwm` and `WorldCoasts.mwm`. -* `-r`: include for each `.mwm` routing index and keep a non routing version as `.mwm.norouting`. +* `-r`: build routing indexes for each `.mwm`. All border polygons from `BORDERS_PATH` are processed into MWM files by default. You can specify only required polygons in `REGIONS` variable, or set it to empty value, so no regular -MWM files are created. The whole process takes 15 hours on a 40-core server, so we suggest +MWM files are created. The whole process takes three days (!) on a 40-core server, so we suggest you specify your e-mail address in a `MAIL` variable and get a mail when the generation is finished. @@ -65,40 +65,38 @@ If a previous run ended with an error, the next one will ignore arguments and co the same arguments as the previous time. Set `-c` option to ignore the stored status. Log files for each region and the entire process (`generate_planet.log`) are written to -`logs` subdirectory of the target. Intermediate data requires around 320 GB of space, and +`logs` subdirectory of the target. Intermediate data requires around 150 GB of space, and to clean it during the process, specify `KEEP_INTDIR=` empty variable. #### Steps The planet generation process is divided in several steps, which are printed during the script run, along with timestamps. To start generation from a specific step, specify it -in the `MODE` variable (and make sure you don't have stored status, or run with `-c` +in the `MODE` variable (and make sure you don't have a stored status, or run with `-c` option). * Step 1 (`coast`): updating planet file. * Step 2: filtering and processing coast lines. If there was a merge error, the script prints way identifiers, waits 40 minutes and tries again from step 1. -* Step R: preparing `.osrm` files for routing indices. PBF files for each region are -cut out of the planet, then OSRM backend scripts process each of these. Can work -asynchronously if `ASYNC_PBF=1` variable is set. * Step 3 (`inter`): generating intermediate data for the planet. * Step 4 (`features`): generating features for each region, splitting the planet. * Step 5 (`mwm`): building the resulting MWMs. -* Step 6 (`routing`): building routing indices out of *.osrm* files. +* Step 6 (`routing`): building routing indices. * Step 7 (`resources`): updating resource and map lists. * Step 8 (`test`): calling `test_planet.sh` to run routing tests. ### Variables -Almost every default in the script can be redefined with environment variables. +Almost any default in the script can be redefined with environment variables. You will have to use some of these, unless you are using our map-building servers. * `GENERATOR_TOOL`: a location of `generator_tool` program. Example: `~/omim-build-debug/out/debug/generator_tool`. * `BUILD_PATH`: a path to either `generator_tool` or its build directory. Example: `~/omim-build-debug`. -* `PLANET`: pa ath or name of the planet file. If there is no file, specify `-U` option, +* `PLANET`: path or name of the planet file. If there is no file, specify `-U` option, and it will be downloaded. Should be in o5m format. Default is `~/planet/planet-latest.o5m`. * `TARGET`: a target path for `mwm` and `routing` files. Some temporary subdirectories will be created inside: `logs`, `borders` and `intermediate_data`. +* `DESC`: a short description of the build, included in all emails. * `MAIL`: comma-separated e-mail addresses, which will receive notifications about finished or failed generation. * `OSMCTOOLS`: a path to pre-compiled OSM C Tools: `osmconvert`, `osmupdate` and @@ -107,10 +105,6 @@ finished or failed generation. script is not moved from `omim/tools/unix`. It is needed for locating a data directory (`omim/data`, can be overridden with `DATA_PATH`), generator tool build path (see `BUILD_PATH`) and OSRM backend scripts (see `OSRM_PATH`). -* `OSRM_PATH`: a path to `omim/3party/osrm/osrm-backend`. Needed for searching -for a routing profile (`$OSRM_PATH/profiles.car.lua` by default, but can be -overridden with `PROFILE`) and for osrm executables (`$OSRM_PATH/build`, -or set `OSRM_BUILD_PATH`). * `DATA_PATH`: a path to classificators and border polygons; the latter can be redefined with `BORDERS_PATH`. * `INTDIR`: a temporary directory that is created when the script starts, and @@ -124,23 +118,18 @@ file, built on step 2, which is required for generating coastlines (step 4). is the default), or on disk (`map`). Tests show that for a complete world, with `map` the process eats some hundreds of gigabytes and crashes, while with `mem` the memory consumption is stable at around 40 GB. -* `ASYNC_PBF`: by default, pbf files for routing are built between steps 2 and 3, -but if this flag is set (e.g. to `1`), they are built asynchronously. But -it can fail due to low memory. * `MERGE_INTERVAL`: delay in minutes between attempts to merge a coast line. Default is 40. * `REGIONS`: a list of `.poly` files for regions to be built. One for each line. +Can be empty. Example: `$(ls ../../data/borders/{UK*,Ireland}.poly)`. * `DELTA_WITH`: a path to an older map directory, to compare with the freshly generated data in the testing step. -Can be empty. Example: `$(ls ../../data/borders/{UK*,Ireland}.poly)`. +* `OLD_INTDIR`: a path to an older `intermediate_data` directory, in case anything +fails. * `OSRM_URL`: address of an OSRM server to calculate worldwide routes. * `SRTM_PATH`: a path to `*.zip` files with SRTM data. * `OSC`: a path to an osmChange file to apply after updating the planet. -* `BOOKING_FILE`: a path to hotels.csv with booking data. -* `BOOKING_USER` and `BOOKING_PASS`: user name and password for booking.com API -* `OPENTABLE_FILE`: a path to restaurants.csv with opentable data. -* `OPENTABLE_USER` and `OPENTABLE_PASS`: user name and password for opentable.com API -to download hotels data. +* `SKIP_TESTS`: if `1`, the testing step is skipped. ### Testing @@ -152,4 +141,4 @@ Run the script with options for `generate_planet`, e.g. omim/tools/unix/islands.sh -lwr In a half an hour you'll get files for 4 regions in a `target` subdirectory. Note that -you can build both generator tool and OSRM backend with the `build_omim.sh` script. +you can build the generator tool with the `build_omim.sh` script. |