path: root/README

Arching Kaos Tools
==================

> Warning: The tools and the whole project is in an experimental state.
> Everything is subject to change.

> Hint: Find the "Script usage" section for sum up of the scripts

Introduction
------------
Tools that help you to create a blockchain called `zchain` by preparing small
JSON objects that are pointing one to another in the following way:
```
ZBLOCK = BLOCK + SIGNATURE
BLOCK = ACTION + DATA + DATA_SIGNATURE + TIMESTAMP + GPG + PREVIOUS_ZBLOCK
DATA = IPFS_CID + SIGNATURE ( + key-pair values that are based on ACTION )
```
Most of the times, IPFS_CID in DATA is expected to be a file. And the module
is responsible for handling it properly.

For the zchain to be valid, all the signatures must be verifiable against the
provided GPG key that is included in the BLOCK.

Previous developments
---------------------
1. A proof of work blockchain is introduced to synchronize the zchains among
   them.
2. A miner prototype is introduced as well.
3. From a directory full of blocks, find the latest block of the longest chain
4. API for calling some of the bash scripts


Description
-----------
This is an installable repository which provides various tools for running
Arching Kaos and using it.

Generic tools:
- `ak` - The tools wrapper
- `ak zchain --crawl` - Can be used to crawl zchains.
- `ak zchain --rebase` - Can be used to rebase your zchain to a previous zblock.
- `ak zchain --reset` - Can be used to reset your zchain to zgenesis.

Modules:
You can use previously known modules or make new ones and put them into your
`modules/` directory. The tree has to be the bare minimum of:
```
modules/
└─ <module name>
    ├─ lib.sh
    └─ main.sh
```
- `mixtapes` and `news` - Can be used to add content to your zchain.
- `profile` - Can be used to build up a profile of the contributor.

Based on your GPG, zchain and other values provided by `profile`, you can create
with `akconfig` a pointer to some basic information that you can share with
others in order for them to crawl your zchain or view your "profile".

There are more tools available under the `bin` folder. Make sure you deeply
understand what they are doing before using them.

Changes are mentioned in `git log` of the repo, as well as in CHANGELOG file.

Requirements
------------
Some Linux machine will propably work. It's tested under Fedora Linux. External
programs used are:

- bash (v5.1.8)
- gpg (v2.3.4)
- wget (v1.21.2)
- curl (v7.79.1)
- git (v2.34.1)
- which (v2.21)
- jq (v1.6)
- nodejs (v18.14)
- npm (v9.5.0)
- npx (v9.5.0)

How to install the repository
-----------------------------

Write on your terminal:
```
git clone https://github.com/arching-kaos/arching-kaos-tools
cd arching-kaos-tools
./install.sh
```

If you encounter any errors, please open an issue.

For more extended description on the installation process, refer to `INSTALL`
file.

Update
------

Navigate to your cloned repository and execute the following commands:

```
git pull
./update.sh
```
Uninstall
---------

Navigate to your cloned repository and execute the following command:

```
./uninstall.sh
```
This will output an archive with your aknet-gpg-keyring to your $HOME folder.

It will NOT remove your IPFS repository, neither is going to clean it.

Examples
--------

### Add a news article ( `ak -m news --add` )

You could use ZCHAIN with NEWS model. Or MIXTAPE model, or make your own.
``` bash
$ ak -m news --create
```

This would pop up a vim editor for you to write a news article or whatever is
text or markdown format with a title.

Saving the file, will save it locally, add it to IPFS, sign it, pack detached
signature with metadata on a JSON object. Then a block will be created packing
your GPG public key, the news/add action with the JSON object and a detached
signature of this, timestamp and finally an entry for the previous *zblock*.
After that, we finally write this as a JSON object, add it to IPFS, sign it
and pack a *ZBLOCK*. The ZBLOCK is then published over our IPNS zchain key.

Other options... let's try help!

``` console
$ ak -m news -h
ak-news - Module to read, create and add zblocks
================================================
    -h, --help                  Prints this help message
    -l, --local-index           Prints an indexed table of your news files
    -i, --import <file>         #TODO
    -a, --add <file>            Creates a data file from the news file you point to
    -r, --read <zblock>         Reads a zblock as a news data
    -c, --create                Vim is going to pop up, you will write and save your
                                newsletter and it's going to be saved
    -s, --specs                 Print specs of data block
```

Clearly there is a TODO item. Import is not working so avoid it, or fix it.
Add is nice, you can add an already existing file directly. `news` is the second
module after `mixtape`. Both modules need refactoring but they work at a level
that someone can be productive with these tools.

### Explore chains ( `ak-enter` )

You can view your zchain as a JSON object using `enter`. There are some flags
in order to either view other zchains or change the depth of view ( includes or
ignores data object and action ).

``` console
$ ak enter -h
ak-enter - Crawl an arching kaos chain
======================================
ak-enter [-N | --no-verify] [-l | --limit <number>] [zblock]
ak-enter [-N | --no-verify] [-l | --limit <number>] -n <zchain>
Usage:
    --help, -h                             Print this help and exit
    --chain <ipns-link>, -n <ipns-link>    Crawl specified chain
    --no-verify, -N                        Don't verify signatures
    <ipfs-link>                            Specify IPFS CID for entrance

Note that combined flags don't work for now
Running with no flags crawls your chain based on AK_ZLATEST environment variable
```

Podman (or Docker)
------------------

There is a ContainerFile that you can use to build an image which you can then
deploy in a container.

Use:

```
podman build -f ContainerFile -t arching-kaos-tools .
```

Specialized ContainerFiles are available in `./podman/` for various
distributions:
- alpine
- archlinux
- debian
- fedora
- opensuse

More over there are two scripts for testing with podman:
1. `./test_with_podman.sh` - build one or all images
2. `./full_test_with_podman.sh` - build one image and bash in it

TODO
----

Next things to come are:
- [ ] - zblock manipulator to fix wrong previous block references and repack
        zblocks.
- [X] - zchain rebase-like procedure to move the zblocks to another seed or set
        another previous block (e.g. join chains)
- [x] - Clean up installation and filesystem usage
- [x] - Log rotate to gzip archives
- [ ] - Filters for log searching for IPFS hashes or names
- [ ] - Use of IPFS file system to store/pin sub chains, previous chains or
        other chains

Concepts under thought
----------------------

- Base chain, derived from personal ones.
- Daemon to monitor chains' states, update and sign new zblocks.
- Find a minimum agreement of what is valid or not. (important)
- Spam defence (e.g. how to protect the network from abuse cases) (important)
- Reduce size of all parts of chain (stringify JSON) (improvement)

Script usage
------------

Utilities
---------
 - ak-log                  # Logging tools
 - ak-clean                # Cleans up temporary folders created by these tools
 - ak-config               # Publish your configuration to IPNS
 - ak-zblock               # zblock tools
 - ak-zchain               # zchain tools
 - ak-data-expand          # Expands and verifies DATA blocks
 - ak-enter                # Crawls ZCHAINS
 - ak-node-info            # Returns IPFS CID or IPNS key for your online config
 - ak-get-gpg              # Return your GPG key as IPFS CID
 - ak-get-zlatest          # Returns zlatest ZBLOCK according to IPFS FS

Schain tools
------------
 - ak-mempool                      # TODO
 - ak-miner-script             PROTO Prototype miner
 - ak-network                      # Network Tools (connect to, scan for, dump peers)
 - ak-sblock-show                  # Returns a mined SBLOCK
 - ak-find-latest-mined-sblock     # In a hay of SBLOCKS finds the latest

Network scouting
----------------
 - ak-stellar-get-participants     # Returns a list of stellar address trusting ARCHINGKAOS

Modules
-------
 - articles                 PROTO Adds an article to your ZCHAIN
 - categories               PROTO Adds or references categories to the ZCHAIN
 - comments                     # References a comment to a ZBLOCK
 - files                        # Adds a file to the ZCHAIN
 - folders                  PROTO Adds a folder to the ZCHAIN
 - mixtapes                     # Adds a mixtape to the ZCHAIN
 - news                         # Adds a news article to the ZCHAIN
 - profile                      # Adds key-value pairs to the ZCHAIN
 - reference                    # Adds references to the ZCHAIN
 - repositories                 # Adds repositories to the ZCHAIN
 - roadmap                  PROTO Adds a roadmap to the ZCHAIN
 - transactions                 # Prototype of transactions in the ZCHAIN
 - sm-files                     # Adds a split file's map to the ZCHAIN
 - todos                    PROTO Adds todo list to the ZCHAIN

 Read more at the `./MODULES` file

Public following (uses ak-profile module)
-----------------------------------------
 - ak-follow               # Follow something
 - ak-following            # List your follows
 - ak-unfollow             # Unfollow something

Zchain tools
------------
 - ak-zchain-chk                   # TODO
 - ak-zchain-rebase                # Rebases ZCHAIN to specified ZBLOCK
 - ak-zchain-reset                 # Resets ZCHAIN to GENESIS
 - ak-zchain-extract-cids          # Retuns CIDs from a zchain
 - ak-zchain-extract-data-cids     # Retuns CIDs of "ipfs" DATA
 - ak-zchain-calculate-size        # Calculates the overall size of your ZCHAIN

File splitters
--------------
 - ak-sm-filejoiner            # Joins a file from a map
 - ak-sm-filesplitter          # Splits a file to chunks and creates a map

FS tools
--------
 - ak-fs

```bash
$ ak fs --help

Tools to add, get and cat files to/from the Arching Kaos File System

Usage:

   -h, --help                                Prints this help message

   --add, --import <file>                    Adds/Imports a file to the AKFS system

   --get, --export <hash> <output file>      Exports a file from the AKFS system

   --cat <hash>                              Concatenates from given hash
```

Libraries
---------
 - lib/_ak_ipfs
 - lib/_ak_fs
 - lib/_ak_gpg
 - lib/_ak_newline
 - lib/_ak_network
 - lib/_ak_script
 - lib/_ak_log

IPFS Wrappers
-------------
 - ak-ipfs-starter
 - ak-ipfs-daemon

Not completed or just odd
-------------------------

Deprecated
----------
 - ak-json2bash


Debugging
---------

Setting `AK_DEBUG` shell variable to `"yes"` will make all log messages to
appear on your stderr (file descriptor 2). After setting this variable, run the
tools you want.

Example:
``` console
AK_DEBUG="yes"; ak zchain --crawl -l 1
```