New post

1 files changed, 366 insertions, 0 deletions

diff --git a/news/1687879381-developments_of_arching_kaos b/news/1687879381-developments_of_arching_kaos
new file mode 100644
index 0000000..88f49e4
--- /dev/null
+++ b/news/1687879381-developments_of_arching_kaos
@@ -0,0 +1,366 @@
+# Up till now [2023.06.22]
+
+We have a ZCHAIN for each contributor
+Each ZCHAIN is referenced from its latest block ZLATEST, zchain-key ZCHAIN, or
+gpg-key GPG_KEY. All three are considered part of the arching-kaos-id AKID which
+is what is published or exchanged between nodes/people.
+
+For example, currently at https://arching-kaos.net we search in the background
+for addresses in the Stellar Network that trust ARCHINGKAOS and have configured
+a variable `config` to an IPNS link that link to an AKID.
+
+Knowing were the ZCHAIN starts, helps us retrieve the ZLATEST block which we are
+calling ZBLOCK. Inside of it, there is a JSON object, consisting of two keys:
+1. "block" and
+2. "block_signature"
+
+Both of these, have values that are formatted as IPFS v0 CID. "block" points to
+another JSON object and "block_signature" points to a GPG armored detach
+signature of BLOCK file.
+
+Inside BLOCK JSON object, there are the following:
+1. "action",
+2. "timestamp",
+3. "data",
+4. "data_signature",
+5. "gpg" and
+6. "previous"
+
+1. ACTION
+It contains a string value. It can be written as a path but describing an action
+for the object in DATA. Currently there is "news/add", "mixtapes/add" and some
+more. Because of that, we can also parse this string as MODULE/ACTION.
+
+2. TIMESTAMP
+It contains a string value. The format should be in UTC seconds as in `date -u +%s`.
+
+3. DATA
+It contains an IPFS v0 CID of a file which is a JSON object. This also contains
+the following keys:
+ 1. "ipfs"
+ 2. "detach"
+ 3. "timestamp" (optional)
+ 4. "filename"
+
+ 1. IPFS
+    It contains an IPFS v0 CID of a file that we want to host in our chain. It
+    can be a text file or anything else.
+ 2. DETACH
+    It contains an IPFS v0 CID of a GPG armored detached signature for the IPFS
+    file above.
+ 3. TIMESTAMP
+    It contains a string value. The format should be in UTC seconds as in
+    `date -u +%s`. However, this is optional field.
+ 4. FILENAME
+    Another optional field. It can be the name of the file if we want to refer
+    to it with a name.
+ Some ACTIONS come with custom DATA JSON objects and maybe adding more keys to
+ the default DATA JSON object. For example, "title", "artist" and "duration".
+ Missing an ACTION "decoder" we just need to verify the signature of the file.
+
+
+
+4. DATA_SIGNATURE
+It contains an IPFS v0 CID of a GPG armored detached signature for the DATA
+file.
+
+5. GPG
+It contains an IPFS v0 CID of a GPG armored public key. It should be the one that
+signs the DATA and BLOCK.
+
+6. PREVIOUS
+It contains an IPFS v0 CID of the previous ZBLOCK of the particular ZCHAIN we
+are on.
+
+# Latest experiments
+
+- ak-sm-filesplitter
+- ak-sm-filejoiner
+These tools above can be used to split a file into 1024byte pieces.
+`ak-sm-filesplitter` will split it and store its pieces renamed to each piece's
+SHA512 hash. It also produces a text file that lists each piece's hash and a
+preformatted filename, along with the original file's SHA512 hash and its
+original filename. The file is called MAP
+`ak-sm-filejoiner` would get this MAP and reassemble the file into your current
+directory.
+
+Adding these two tools, came with the introduction of `ak-sm-files` script.
+This script will take this map, add its hash to the "shamap" key in the DATA
+JSON object, as well as its GPG armored detached signature in "shamapsig".
+Currently though, the "ipfs" key has the IPFS v0 CID of the original file which
+is subject to change.
+
+Furthermore, with the introduction of these tools, ZCHAIN could point to SHA512
+hashes that point to maps instead of IPFS v0 CIDs.
+
+MAP contains X+1 MAPLINES. A MAPLINE is consisting from a SHA512 hash, two
+spaces and a filename up to 254 characters. This filename hosts also a piece
+number. The +1 line is the SHA512 hash of the original hash, two spaces and the
+original filename.
+
+However, this is the initial design of the `ak-sm-*` "series".
+
+Now we have to make the MAP just a pointer to a MerkleTree based on SHA512 hash.
+We should keep that last line, but instead of the original hash, we provide the
+root of the MerkleTree of that file.
+
+# Big picture
+
+Well, this is another part. Experimentaly, there are some tools introduced.
+These are used in the following manner and with the following purpose.
+
+We introduced a SCHAIN. It consists from the following parts:
+1. zblocks
+2. zpairs
+3. reward
+4. miner
+5. previous
+6. nonce
+7. timestamp
+
+After considering the above as already implemented in a way, it could be of use
+to add one more
+8. blocknumber
+
+But before adding, let's see what's already there.
+
+  1. ZBLOCKs
+  Array of IPFS v0 CIDs that point to individual ZBLOCKs. These has to be valid
+  ZBLOCK that is properly signed and goes back to a valid or broadly acceptable
+  GENESIS block.
+  2. ZPAIRs
+  Array of JSON objects. Each object consists of 2 keys. "zchain" and "zlatest".
+  The "zchain" part is an IPNS key and "zlatest" is an IPFS v0 CID pointer to a
+  ZBLOCK, supposingly the ZLATEST of the ZCHAIN.
+  3. REWARD
+  Reward is a supposed retribution for the contribution of the miner that made
+  up this SBLOCK.
+  4. MINER
+  This is the address of the miner. It is a IPFS v0 CID pointing to a GPG
+  armored public key.
+  5. PREVIOUS
+  This is an SHA512 hash value which is the hash of the previous SBLOCK.
+  6. NONCE
+  Nonce is a random number that when is added in the forming SBLOCK would result
+  a file whose hash would start with some acceptable value. In our example, the
+  difficulty of the problem is on finding a SBLOCK whose hash starts with 3
+  zeros.
+  7. TIMESTAMP
+  As previously, `data -u +%s` date and time format. -u stands for UTC time.
+
+  Now we can say that we would like also the eighty proposed key:
+  8. BLOCK_NUMBER
+  This should be start counting from 1 with PREVIOUS pointing to an agreed
+  GENESIS block.
+
+# About GENESIS block of the SCHAIN
+
+A nice GENESIS block would be nice to consist of an outline of the whole design.
+It would also be nice to host on some ZCHAIN a couple of tools as well.
+
+# Things that should be figured out about SCHAIN
+
+1. From where the REWARD is verified?
+
+We could have either a static REWARD or a scaling one.
+
+2. Limit of total REWARD amount can be in circulation?
+
+As of creation of ARCHINGKAOS in Stellar Network is 10000 with 7 decimal digits,
+we would say that in integer value we can have as much as 100,000,000,000. That
+would be the limit.
+
+3. How a transaction can be considered valid?
+
+Transactions are part of the ZCHAIN implementation. In order to make use of them
+in a valid way, you should have rewards or previously transfered REWARDs from
+someone that had also either won them or previously transfered to them, etc.
+
+Also, this has to have happened at least 6 blocks ago, just to ensure that the
+SBLOCK containing the enabling transaction is final and well established.
+
+4. Do we have to collect fees? From where and how?
+
+Possibly, yes. In a bitcoin manner. We could "sell" the bytes that someone is
+posting with announcing their zblock or zchain link.
+
+The miners of the network should validate the fee amounts, the size of the
+ZBLOCK plus the size of the in between ZBLOCKS until the last one already known.
+
+The fees should be calculated by the contributor of the ZCHAIN. It should be
+also the final ZBLOCK.
+
+A fee zblock could be like this:
+```
+{
+    "id": "1",
+    "zblock": "QmTkgtbZPGFyLHz1eigYQGLMiijPh3TqkAemi9gxBNZFVo",
+    "block": "QmaYFNVuEMnFH2fG5Czz7pWzJC1PX23d9RnyowcGgogyhY",
+    "timestamp": "1686961638",
+    "block_signature": "QmZyTrMn286zeGAZyr9ncoLKdSP5VACeQBtxQY5ksxQ2qL",
+    "detach": "QmZcbxbnXkCQu6aeZCu83dUxNQ6khETDxVn54TmLMDGid5",
+    "module": "pay",
+    "action": "fee",
+    "gpg": "QmaNjKETyxBGKVnAuV9iUkbTge5cFf4m8K1sekCeKMRAL5",
+    "data": "QmUtJypVap2zZPnSd7ws32kUhVmtvDVheWNKMtLV3RqiJh",
+    "QmUtJypVap2zZPnSd7ws32kUhVmtvDVheWNKMtLV3RqiJh": {
+        "fee": "100",
+        "ipfs": "QmVfPep9KcczZwgvufRTnz9LFFFWpEjmFNj5NrMXJxL4Me",
+        "detach": "QmVE4Ejb6hdErXrJmkjPjPequZJfpwQnuvUiXYJiyyPX4b"
+    },
+    "previous": "QmeScAUMNeBYJanAuSTRo9RZi4QCdJuD8xdrS5SniygDPH"
+}
+```
+
+
+However, this means that the fee ZBLOCK alone costs 654 without even considering
+the sizes of the block_signature, data_signature, ipfs and detach signature. The
+other solution would be to just add a "fee" key in our BLOCK. But we can also
+use the before mention method to pay for multiple ZBLOCKS.
+
+
+5. Who holds the REWARDs and gets the FEEs as well?
+
+We will baptize a system as "coinbase" which would be responsible for gathering
+the fees and giving out the relative rewards per mined SBLOCK to the miner. If
+the coinbase is not satisfied with a proposed SBLOCK, then it gets rejected.
+
+
+# Abstraction
+
+## About SBLOCK
+Since almost everything is a sha512sum and the SBLOCK can have 8 keys, we could
+remove the tags and set a format that would sum up to 1024 bytes.
+1. MerkleTree to a list of ZBLOCKs
+2. MerkleTree to a list of ZPAIRs
+3. MerkleTree to the public GPG key of the MINER
+4. MerkleTree of the PREVIOUSly mined SBLOCK
+
+Non SHA512 values:
+5. REWARD amount
+6. NONCE value added to mine successfully
+7. TIMESTAMP in UTC
+8. BLOCKNUMBER to just know where we are
+
+## Revisiting ZBLOCK structure
+
+ZBLOCK
+IPFS v0 CID -> SHA512 hash of a file containing... the following
+
+ZBLOCK inside
+```
+{
+    "block": "IPFS v0 CID -> SHA512 hash",
+    "block_signature": "IPFS v0 CID -> SHA512 hash"
+}
+```
+BLOCK inside
+```
+{
+    "id": "1",
+    "timestamp": "1686961638",
+    "detach": "IPFS v0 CID -> SHA512 hash",
+    "module": "pay",
+    "action": "fee",
+    "gpg": "IPFS v0 CID -> SHA512 hash",
+    "data": "IPFS v0 CID -> SHA512 hash",
+    "previous": "IPFS v0 CID -> SHA512 hash"
+}
+```
+A proposal also would be to remove "action" and "module" and replace it with
+the MerkleTree root of the script made the block.
+
+This script would address creation and handling of the "data" key and furtherly
+JSON object.
+
+Note also that "module" is NOT declared directly in its own key, but it is
+extracted from the first part of the "action" key, when key is in the form of
+"module/action" format.
+
+By changing the "action" to splitted "module" "action", we can then use "module"
+to declare our script that reads the DATA JSON object.
+
+Putting, though, a script that can be possibly vulnurable or malicious raises
+questions. How we can implement a decoder script that we can safely execute in
+our system? How we protect from eg a wallet stealer or other malware related
+scripts?
+
+Isolated environment?
+
+Signed modules?
+Accept signed modules from specific public GPG keys that should be declared in
+the GENESIS block of the SCHAIN initially. After the initial GENESIS
+declaration, initial module contributors, could sign new contributors' scripts
+or people could also vote for the feature, or accept it for their node only.
+
+Therefore, ACTION could be performed optionally and accepted by the miners also
+optionally. For example, they could make up a whitelist and/or a blacklist for
+certain scripts.
+
+DATA inside
+
+```
+{
+    "timestamp":"1686961638",
+    "fee": "100",
+    "ipfs": "IPFS v0 CID -> SHA512 hash MerkleTree",
+    "detach": "IPFS v0 CID -> SHA512 hash"
+}
+```
+
+# Disagreements
+
+Talking about the ACTION scripts above, comes with a possibility of disagreement
+in the network. X approves Y but W doesn't. Options:
+
+1. If X is part of the GENESIS block but Y isn't, then W can continue trusting
+X without being obligated to run scripts provided by Y. In that sense, W can
+choose to not propagate Y related ZBLOCKs. Meaning not only ZBLOCKs that are
+announced to the network signed by Y but also other ZBLOCKs that contain actions
+that use Y's script.
+
+W now can choose one of the following:
+1. Fork the chain declaring their disagreement in a ZBLOCK. This should be mined
+by someone or them. Next, people that agree with W should combine their powers
+to continue the fork.
+2. Don't fork but do not keep the relative information. Either the script or the
+ZBLOCKs that are refering to it. However, if this strategy is adopted by
+everyone then the SCHAIN will be not be able to be crawled back. For that could
+leave an instruction to skip the particular SBLOCKs or ZBLOCKs.
+
+# Speaking of crawling back
+
+Because of the design of the ZCHAIN, the tools that got developed were to create
+DATA, BLOCKs and ZBLOCKs with the purpose of getting chained together in one
+ZCHAIN, which would be able to be read by crawling from the latest ZBLOCK to the
+GENESIS block. The opposite direction for retrieving the ZCHAIN is not created.
+
+For that, in a let's say, initial state of our node, we could just ask other
+nodes for the current hash of the SCHAIN. From there, we could crawl back to the
+GENESIS block of the SCHAIN. This should be done first to index the SCHAIN.
+
+After that, we only have ( SBLOCK_AMOUNT + GENESIS ) * 1024 bytes of data and a
+lot of work to do with these. If we were about to calculate that each new SBLOCK
+is mined in around 1 minute, then for a year we have around 500MB for storing
+just SBLOCKs. With these data, we can calculate if the amount of blocks is close
+to the expected block amount, given the declarations in GENESIS block.
+
+```
+// Timestamps are in seconds
+expected_block_number_now = floor((now_timestamp - genesis_timestamp) / 60 )
+```
+
+# Other things to consider
+
+No idea, the train of thought reach its final destination. We will be turning it
+around and be back with an even better overview and probably with some of the
+above mentioned features, implemented or something.
+
+For sure, this document could be splitted in a way so it can fit in a 
+comprehensive way into proposals in the appropriate repository.
+
+Other than that, if you really reached this lines, a huge thank you.
+
+More than happy to hear any feedback from you at the IRC place:
+https://irc.arching-kaos.net
+