path: root/news/1687879381-developments_of_arching_kaos

# Up till now [2023.06.22]

We have a ZCHAIN for each contributor
Each ZCHAIN is referenced from its latest block ZLATEST, zchain-key ZCHAIN, or
gpg-key GPG_KEY. All three are considered part of the arching-kaos-id AKID which
is what is published or exchanged between nodes/people.

For example, currently at https://arching-kaos.net we search in the background
for addresses in the Stellar Network that trust ARCHINGKAOS and have configured
a variable `config` to an IPNS link that link to an AKID.

Knowing were the ZCHAIN starts, helps us retrieve the ZLATEST block which we are
calling ZBLOCK. Inside of it, there is a JSON object, consisting of two keys:
1. "block" and
2. "block_signature"

Both of these, have values that are formatted as IPFS v0 CID. "block" points to
another JSON object and "block_signature" points to a GPG armored detach
signature of BLOCK file.

Inside BLOCK JSON object, there are the following:
1. "action",
2. "timestamp",
3. "data",
4. "data_signature",
5. "gpg" and
6. "previous"

1. ACTION
It contains a string value. It can be written as a path but describing an action
for the object in DATA. Currently there is "news/add", "mixtapes/add" and some
more. Because of that, we can also parse this string as MODULE/ACTION.

2. TIMESTAMP
It contains a string value. The format should be in UTC seconds as in `date -u +%s`.

3. DATA
It contains an IPFS v0 CID of a file which is a JSON object. This also contains
the following keys:
 1. "ipfs"
 2. "detach"
 3. "timestamp" (optional)
 4. "filename"

 1. IPFS
    It contains an IPFS v0 CID of a file that we want to host in our chain. It
    can be a text file or anything else.
 2. DETACH
    It contains an IPFS v0 CID of a GPG armored detached signature for the IPFS
    file above.
 3. TIMESTAMP
    It contains a string value. The format should be in UTC seconds as in
    `date -u +%s`. However, this is optional field.
 4. FILENAME
    Another optional field. It can be the name of the file if we want to refer
    to it with a name.
 Some ACTIONS come with custom DATA JSON objects and maybe adding more keys to
 the default DATA JSON object. For example, "title", "artist" and "duration".
 Missing an ACTION "decoder" we just need to verify the signature of the file.



4. DATA_SIGNATURE
It contains an IPFS v0 CID of a GPG armored detached signature for the DATA
file.

5. GPG
It contains an IPFS v0 CID of a GPG armored public key. It should be the one that
signs the DATA and BLOCK.

6. PREVIOUS
It contains an IPFS v0 CID of the previous ZBLOCK of the particular ZCHAIN we
are on.

# Latest experiments

- ak-sm-filesplitter
- ak-sm-filejoiner
These tools above can be used to split a file into 1024byte pieces.
`ak-sm-filesplitter` will split it and store its pieces renamed to each piece's
SHA512 hash. It also produces a text file that lists each piece's hash and a
preformatted filename, along with the original file's SHA512 hash and its
original filename. The file is called MAP
`ak-sm-filejoiner` would get this MAP and reassemble the file into your current
directory.

Adding these two tools, came with the introduction of `ak-sm-files` script.
This script will take this map, add its hash to the "shamap" key in the DATA
JSON object, as well as its GPG armored detached signature in "shamapsig".
Currently though, the "ipfs" key has the IPFS v0 CID of the original file which
is subject to change.

Furthermore, with the introduction of these tools, ZCHAIN could point to SHA512
hashes that point to maps instead of IPFS v0 CIDs.

MAP contains X+1 MAPLINES. A MAPLINE is consisting from a SHA512 hash, two
spaces and a filename up to 254 characters. This filename hosts also a piece
number. The +1 line is the SHA512 hash of the original hash, two spaces and the
original filename.

However, this is the initial design of the `ak-sm-*` "series".

Now we have to make the MAP just a pointer to a MerkleTree based on SHA512 hash.
We should keep that last line, but instead of the original hash, we provide the
root of the MerkleTree of that file.

# Big picture

Well, this is another part. Experimentaly, there are some tools introduced.
These are used in the following manner and with the following purpose.

We introduced a SCHAIN. It consists from the following parts:
1. zblocks
2. zpairs
3. reward
4. miner
5. previous
6. nonce
7. timestamp

After considering the above as already implemented in a way, it could be of use
to add one more
8. blocknumber

But before adding, let's see what's already there.

  1. ZBLOCKs
  Array of IPFS v0 CIDs that point to individual ZBLOCKs. These has to be valid
  ZBLOCK that is properly signed and goes back to a valid or broadly acceptable
  GENESIS block.
  2. ZPAIRs
  Array of JSON objects. Each object consists of 2 keys. "zchain" and "zlatest".
  The "zchain" part is an IPNS key and "zlatest" is an IPFS v0 CID pointer to a
  ZBLOCK, supposingly the ZLATEST of the ZCHAIN.
  3. REWARD
  Reward is a supposed retribution for the contribution of the miner that made
  up this SBLOCK.
  4. MINER
  This is the address of the miner. It is a IPFS v0 CID pointing to a GPG
  armored public key.
  5. PREVIOUS
  This is an SHA512 hash value which is the hash of the previous SBLOCK.
  6. NONCE
  Nonce is a random number that when is added in the forming SBLOCK would result
  a file whose hash would start with some acceptable value. In our example, the
  difficulty of the problem is on finding a SBLOCK whose hash starts with 3
  zeros.
  7. TIMESTAMP
  As previously, `data -u +%s` date and time format. -u stands for UTC time.

  Now we can say that we would like also the eighty proposed key:
  8. BLOCK_NUMBER
  This should be start counting from 1 with PREVIOUS pointing to an agreed
  GENESIS block.

# About GENESIS block of the SCHAIN

A nice GENESIS block would be nice to consist of an outline of the whole design.
It would also be nice to host on some ZCHAIN a couple of tools as well.

# Things that should be figured out about SCHAIN

1. From where the REWARD is verified?

We could have either a static REWARD or a scaling one.

2. Limit of total REWARD amount can be in circulation?

As of creation of ARCHINGKAOS in Stellar Network is 10000 with 7 decimal digits,
we would say that in integer value we can have as much as 100,000,000,000. That
would be the limit.

3. How a transaction can be considered valid?

Transactions are part of the ZCHAIN implementation. In order to make use of them
in a valid way, you should have rewards or previously transfered REWARDs from
someone that had also either won them or previously transfered to them, etc.

Also, this has to have happened at least 6 blocks ago, just to ensure that the
SBLOCK containing the enabling transaction is final and well established.

4. Do we have to collect fees? From where and how?

Possibly, yes. In a bitcoin manner. We could "sell" the bytes that someone is
posting with announcing their zblock or zchain link.

The miners of the network should validate the fee amounts, the size of the
ZBLOCK plus the size of the in between ZBLOCKS until the last one already known.

The fees should be calculated by the contributor of the ZCHAIN. It should be
also the final ZBLOCK.

A fee zblock could be like this:
```
{
    "id": "1",
    "zblock": "QmTkgtbZPGFyLHz1eigYQGLMiijPh3TqkAemi9gxBNZFVo",
    "block": "QmaYFNVuEMnFH2fG5Czz7pWzJC1PX23d9RnyowcGgogyhY",
    "timestamp": "1686961638",
    "block_signature": "QmZyTrMn286zeGAZyr9ncoLKdSP5VACeQBtxQY5ksxQ2qL",
    "detach": "QmZcbxbnXkCQu6aeZCu83dUxNQ6khETDxVn54TmLMDGid5",
    "module": "pay",
    "action": "fee",
    "gpg": "QmaNjKETyxBGKVnAuV9iUkbTge5cFf4m8K1sekCeKMRAL5",
    "data": "QmUtJypVap2zZPnSd7ws32kUhVmtvDVheWNKMtLV3RqiJh",
    "QmUtJypVap2zZPnSd7ws32kUhVmtvDVheWNKMtLV3RqiJh": {
        "fee": "100",
        "ipfs": "QmVfPep9KcczZwgvufRTnz9LFFFWpEjmFNj5NrMXJxL4Me",
        "detach": "QmVE4Ejb6hdErXrJmkjPjPequZJfpwQnuvUiXYJiyyPX4b"
    },
    "previous": "QmeScAUMNeBYJanAuSTRo9RZi4QCdJuD8xdrS5SniygDPH"
}
```


However, this means that the fee ZBLOCK alone costs 654 without even considering
the sizes of the block_signature, data_signature, ipfs and detach signature. The
other solution would be to just add a "fee" key in our BLOCK. But we can also
use the before mention method to pay for multiple ZBLOCKS.


5. Who holds the REWARDs and gets the FEEs as well?

We will baptize a system as "coinbase" which would be responsible for gathering
the fees and giving out the relative rewards per mined SBLOCK to the miner. If
the coinbase is not satisfied with a proposed SBLOCK, then it gets rejected.


# Abstraction

## About SBLOCK
Since almost everything is a sha512sum and the SBLOCK can have 8 keys, we could
remove the tags and set a format that would sum up to 1024 bytes.
1. MerkleTree to a list of ZBLOCKs
2. MerkleTree to a list of ZPAIRs
3. MerkleTree to the public GPG key of the MINER
4. MerkleTree of the PREVIOUSly mined SBLOCK

Non SHA512 values:
5. REWARD amount
6. NONCE value added to mine successfully
7. TIMESTAMP in UTC
8. BLOCKNUMBER to just know where we are

## Revisiting ZBLOCK structure

ZBLOCK
IPFS v0 CID -> SHA512 hash of a file containing... the following

ZBLOCK inside
```
{
    "block": "IPFS v0 CID -> SHA512 hash",
    "block_signature": "IPFS v0 CID -> SHA512 hash"
}
```
BLOCK inside
```
{
    "id": "1",
    "timestamp": "1686961638",
    "detach": "IPFS v0 CID -> SHA512 hash",
    "module": "pay",
    "action": "fee",
    "gpg": "IPFS v0 CID -> SHA512 hash",
    "data": "IPFS v0 CID -> SHA512 hash",
    "previous": "IPFS v0 CID -> SHA512 hash"
}
```
A proposal also would be to remove "action" and "module" and replace it with
the MerkleTree root of the script made the block.

This script would address creation and handling of the "data" key and furtherly
JSON object.

Note also that "module" is NOT declared directly in its own key, but it is
extracted from the first part of the "action" key, when key is in the form of
"module/action" format.

By changing the "action" to splitted "module" "action", we can then use "module"
to declare our script that reads the DATA JSON object.

Putting, though, a script that can be possibly vulnurable or malicious raises
questions. How we can implement a decoder script that we can safely execute in
our system? How we protect from eg a wallet stealer or other malware related
scripts?

Isolated environment?

Signed modules?
Accept signed modules from specific public GPG keys that should be declared in
the GENESIS block of the SCHAIN initially. After the initial GENESIS
declaration, initial module contributors, could sign new contributors' scripts
or people could also vote for the feature, or accept it for their node only.

Therefore, ACTION could be performed optionally and accepted by the miners also
optionally. For example, they could make up a whitelist and/or a blacklist for
certain scripts.

DATA inside

```
{
    "timestamp":"1686961638",
    "fee": "100",
    "ipfs": "IPFS v0 CID -> SHA512 hash MerkleTree",
    "detach": "IPFS v0 CID -> SHA512 hash"
}
```

# Disagreements

Talking about the ACTION scripts above, comes with a possibility of disagreement
in the network. X approves Y but W doesn't. Options:

1. If X is part of the GENESIS block but Y isn't, then W can continue trusting
X without being obligated to run scripts provided by Y. In that sense, W can
choose to not propagate Y related ZBLOCKs. Meaning not only ZBLOCKs that are
announced to the network signed by Y but also other ZBLOCKs that contain actions
that use Y's script.

W now can choose one of the following:
1. Fork the chain declaring their disagreement in a ZBLOCK. This should be mined
by someone or them. Next, people that agree with W should combine their powers
to continue the fork.
2. Don't fork but do not keep the relative information. Either the script or the
ZBLOCKs that are refering to it. However, if this strategy is adopted by
everyone then the SCHAIN will be not be able to be crawled back. For that could
leave an instruction to skip the particular SBLOCKs or ZBLOCKs.

# Speaking of crawling back

Because of the design of the ZCHAIN, the tools that got developed were to create
DATA, BLOCKs and ZBLOCKs with the purpose of getting chained together in one
ZCHAIN, which would be able to be read by crawling from the latest ZBLOCK to the
GENESIS block. The opposite direction for retrieving the ZCHAIN is not created.

For that, in a let's say, initial state of our node, we could just ask other
nodes for the current hash of the SCHAIN. From there, we could crawl back to the
GENESIS block of the SCHAIN. This should be done first to index the SCHAIN.

After that, we only have ( SBLOCK_AMOUNT + GENESIS ) * 1024 bytes of data and a
lot of work to do with these. If we were about to calculate that each new SBLOCK
is mined in around 1 minute, then for a year we have around 500MB for storing
just SBLOCKs. With these data, we can calculate if the amount of blocks is close
to the expected block amount, given the declarations in GENESIS block.

```
// Timestamps are in seconds
expected_block_number_now = floor((now_timestamp - genesis_timestamp) / 60 )
```

# Other things to consider

No idea, the train of thought reach its final destination. We will be turning it
around and be back with an even better overview and probably with some of the
above mentioned features, implemented or something.

For sure, this document could be splitted in a way so it can fit in a 
comprehensive way into proposals in the appropriate repository.

Other than that, if you really reached this lines, a huge thank you.

More than happy to hear any feedback from you at the IRC place:
https://irc.arching-kaos.net