Skip to content

Say in AGENTS.md how to get the data, build and run the examples #41

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jwrosewell merged 2 commits into from
Nov 4, 2026
Merged

Say in AGENTS.md how to get the data, build and run the examples #41

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d11d85f
Say in AGENTS.md how to get the data, build and run the examples
jwrosewell Nov 4, 2026
a09c5dc
Describe only the cxx/data layout in the AGENTS.md run steps
jwrosewell Nov 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
231 changes: 229 additions & 2 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,226 @@
# Working in this repository

Notes on what is not obvious from the code. Keep them short and current.
Building, testing and the layout are described in the [README](README.md).
If you have been asked to get an example running, follow
[Run the examples](#run-the-examples). It gives every step in order, from a
fresh clone to the output. The sections after it are notes on what is not
obvious from the code, for changing it. Keep them short and current. The
[README](README.md) describes the build, the tests and the layout.

## Run the examples

Run each command from the repository root unless the step says otherwise.
Device Detection's on-premise `GettingStartedOnPrem` is the quickest example
to show, because its data file is about 35 MB. IP Intelligence's on-premise
`GettingStartedOnPrem` needs a download of about 230 MB that unpacks to about
460 MB, so run it second.

The examples use the packages published to Maven Central unless the `dev`
profile is active (see
[One reactor, six package trees](#one-reactor-six-package-trees)). Those
packages carry the native engines, so building only an example folder, as
below, needs no CMake and no C++ toolchain. `mvn install` at the root builds
the native engines from `cxx` and does need them, as the README says.

### Prerequisites

- Git.
- A JDK, 8 or later. CI tests 8, 11, 17 and 21 (`ci/options.json`).
- Maven 3.8 or later, or the `mvnw` wrapper at the root.
- Network access to GitHub, to Maven Central for the packages the examples
use, and to `distributor.51degrees.com` for the data files.
- For the data files, `curl`, `gzip` and `md5sum` (`md5` on macOS) on Linux
and macOS, where the examples download with the shell script, and Windows
PowerShell 5.1 or PowerShell 7 on Windows, where they use the PowerShell
script.

Clone the repository with its submodules.

```
git clone --recurse-submodules https://github.com/51Degrees/51degrees-java.git
cd 51degrees-java
```

In a clone made without `--recurse-submodules`, fetch the submodules.

```
git submodule update --init --recursive
```

### Data files

The examples use the free Lite data files, which need no licence key. They are
not stored in this repository. The `cxx` submodule holds the
[data](https://github.com/51Degrees/data) repository as `cxx/data`, which has a
link to each file, and a script downloads them into `cxx/data/device-detection`
and `cxx/data/ip-intelligence`.

On Linux and macOS, run the shell script.

```
cxx/data/get-data-files.sh
```

On Windows, run the PowerShell script. Windows PowerShell can refuse to run a
script file under the policy Windows sets by default, so run it the way the
examples do.

```
powershell -NoProfile -ExecutionPolicy Bypass -File cxx/data/get-data-files.ps1
```

Where PowerShell 7 is installed, `pwsh cxx/data/get-data-files.ps1` does the
same on any system. Either script downloads every free file, and keeps a file
that is already there. To download one file, give its name, as in
`cxx/data/get-data-files.sh 51Degrees-LiteV4.1.hash`. An example whose free
file is missing runs the script for that file itself.

#### Check the files

- `cxx/data/device-detection/51Degrees-LiteV4.1.hash` is more than 10 MB.
- `cxx/data/ip-intelligence/51Degrees-LiteV41.ipi` is more than 100 MB.

### Build

Build the console examples of a product, with the examples' shared module, in
that product's examples folder. Each build makes a jar that holds the example
and everything it needs.

```
cd examples/device-detection
mvn -pl console -am package -DskipTests
cd ../..
cd examples/ip-intelligence
mvn -pl console -am package -DskipTests
cd ../..
```

With the wrapper, run `../../mvnw` in place of `mvn`, or `..\..\mvnw.cmd` on
Windows. The jars are
`examples/device-detection/console/target/device-detection-java-examples.console-<version>-jar-with-dependencies.jar`
and
`examples/ip-intelligence/console/target/ip-intelligence-java-examples.console-<version>-jar-with-dependencies.jar`,
where `<version>` is the `<version>` in that examples folder's `pom.xml`,
`4.5.127-SNAPSHOT` on `rc` today.

### Run

Run the examples from the repository root. They find `logback.xml`, the data
file and their options file with `FileFinder`, which searches only below the
folder Java runs in (`pipeline/pipeline.core/src/main/java/fiftyone/pipeline/util/FileFinder.java`).
When a free data file is not there, the example finds its `.link` the same way
and runs the download script for it (`findDataFile` in each examples folder's
`shared` module, `DataFileHelper.java`), with Windows PowerShell on Windows and
`sh` on Linux and macOS. The same commands work on Windows, Linux and macOS.

```
java -cp examples/device-detection/console/target/device-detection-java-examples.console-4.5.127-SNAPSHOT-jar-with-dependencies.jar fiftyone.devicedetection.examples.console.GettingStartedOnPrem
```

The IP Intelligence example looks for the paid Enterprise file,
`ip-intelligence/51Degrees-EnterpriseIpiV41.ipi`, so give it the Lite file as
its first argument.

```
java -cp examples/ip-intelligence/console/target/ip-intelligence-java-examples.console-4.5.127-SNAPSHOT-jar-with-dependencies.jar fiftyone.ipintelligence.examples.console.GettingStartedOnPrem cxx/data/ip-intelligence/51Degrees-LiteV41.ipi
```

The example downloads the Lite file first when it is not there. In place of
the argument, the `51DEGREES_IPI_PATH` environment variable or system property
can give the absolute path of a data file that is already there.

On Java 24 or later, add `--enable-native-access=ALL-UNNAMED` before the class
name, as
[examples/device-detection/README.md](examples/device-detection/README.md#native-library-access)
explains, or Java warns about the native library.

### Correct output

When an example downloads its data file, a Logback line first reports
`'<file>' is missing, so it is being downloaded by running` and the command,
and the script's `Downloading` and `Saved` lines follow.

The Device Detection example processes five sets of evidence. For each it
prints `Input values:` and the evidence, then `Results:` with
`Mobile Device`, `Platform Name`, `Platform Version`, `Browser Name` and
`Browser Version`. Logback lines come before and after. The last ones report
the data file and end the run, with these messages.

```
Used a 'Lite' data file, created <date>, <n> days ago, from location '<path>'
This example is using the 'Lite ' data file. This is used for illustration, ...
All done
```

A warning that the data file is more than 28 days old may come too, which is
expected with the Lite file. The exit code is 0.

The IP Intelligence example looks up `116.154.188.222`, `45.236.48.61` and an
IPv6 address. For each it prints `Input values:` with `query.client-ip`, then
`Results:` with a line for each property, from `RegisteredName` to
`TimeZoneOffset`. The Lite file has RegisteredCountry, RegisteredName and
RegisteredOwner, so a line for a property it does not have reads
`Unknown. The property '<name>' is not available in this data file.` followed
by a link, and the first such line also logs a warning that some properties
need a paid subscription. It then logs the data file in the same way, and
`All done`.

The paid data files have many more properties, as the
[pricing page](https://51degrees.com/pricing?utm_source=github&utm_medium=docs&utm_campaign=51degrees-java&utm_content=agents.md&utm_term=correct-output)
describes.

### Cloud examples

The cloud examples need a resource key, which a person creates in the
configurator, so ask for one if none is set. A key with the properties most of
the Device Detection examples use can be made
[here](https://configure.51degrees.com/jqz435Nc?utm_source=github&utm_medium=docs&utm_campaign=51degrees-java&utm_content=agents.md&utm_term=cloud-examples).
The Device Detection cloud example takes the key as its first argument, or
from a `TestResourceKey` environment variable or system property.

```
java -cp examples/device-detection/console/target/device-detection-java-examples.console-4.5.127-SNAPSHOT-jar-with-dependencies.jar fiftyone.devicedetection.examples.console.GettingStartedCloud <key>
```

It prints the same `Input values:` and `Results:` blocks as the on-premise
example. The IP Intelligence cloud examples read their key from
`51DEGREES_RESOURCE_KEY` first and `TestResourceKey` second, and
`examples/ip-intelligence/README.md` gives the keys to create for them.

### Pitfalls

- **A data file that is not there.** `cxx/data` holds a `.link` for each file
rather than the file. An example whose file is missing runs the download
script for it, so it needs the network and the tools under "Prerequisites".
Check the file sizes as above.
- **Missing submodules.** The example then logs `Could not find the data file
'device-detection/51Degrees-LiteV4.1.hash' which must be somewhere in the
project space to be found.` Run `git submodule update --init --recursive`. A
zip download of the repository has no submodules, so clone with Git.
- **Running from another folder.** `FileFinder` looks only below the folder
Java runs in, so `java` started in `examples/device-detection`, as the
examples' README shows, cannot find the data file or its link in `cxx`.
Start it from the repository root. The same applies to `mvn exec:java`,
which runs in the folder Maven starts in.
- **The version in the jar name.** The examples' READMEs show jar names with
`4.4.20` and `4.4.19`, which are out of date. Use the name in `target`.
- **The IP Intelligence example without an argument.** It searches for the
Enterprise file, logs `Failed to find IP Intelligence data file` and stops,
even when the Lite file is in place.
- **The `dev` profile.** `-Pdev`, which CI uses, builds the examples against
the libraries here, which needs `mvn install` at the root first and so the
native build, with CMake 3.10 or later and a C++ toolchain. It is not needed
to run the examples.
- **Platforms the packages cover.** The published
`device-detection.hash.engine.on-premise` 4.5.128 jar has native libraries
for Windows x64 and x86, Linux x64 and aarch64, and macOS x64 and aarch64.
The `ip-intelligence.engine.on-premise` 4.5.128 jar has the same, without
Windows x86. On a platform a jar lacks, the example cannot load the engine.
- **What CI does not test.** On a pull request CI builds and tests the
examples with the `dev` profile (`ci/build-project.ps1` and
`ci/run-unit-tests.ps1`), and against the published packages only when it
publishes (`ci/run-integration-tests.ps1`). It never runs the console
example jars as above, and the example tests use the paid data files when
CI has them.

## One reactor, six package trees

Expand Down Expand Up @@ -44,6 +263,14 @@ The Windows pre-build scripts take a toolset and SDK version from the pom.
JavaScript templates and the OWID library are submodules inside `pipeline`,
where the Pipeline API expects them.

## Releases

Pull requests go to `rc`, not `main`, and stay drafts until they have been
reviewed, because the nightly merges any pull request into `rc` that is not a
draft and has no review requested. When a release is wanted, one pull request
merges `rc` into `main`, and the "Only rc into main" check fails a pull request
into `main` from any other branch.

## Writing

Plain words, no em dashes, sentence case headings. Nothing in this public
Expand Down
Loading