diff options
Diffstat (limited to 'news/1687879381-developments_of_arching_kaos')
| -rw-r--r-- | news/1687879381-developments_of_arching_kaos | 366 |
1 files changed, 366 insertions, 0 deletions
diff --git a/news/1687879381-developments_of_arching_kaos b/news/1687879381-developments_of_arching_kaos new file mode 100644 index 0000000..88f49e4 --- /dev/null +++ b/news/1687879381-developments_of_arching_kaos @@ -0,0 +1,366 @@ +# Up till now [2023.06.22] + +We have a ZCHAIN for each contributor +Each ZCHAIN is referenced from its latest block ZLATEST, zchain-key ZCHAIN, or +gpg-key GPG_KEY. All three are considered part of the arching-kaos-id AKID which +is what is published or exchanged between nodes/people. + +For example, currently at https://arching-kaos.net we search in the background +for addresses in the Stellar Network that trust ARCHINGKAOS and have configured +a variable `config` to an IPNS link that link to an AKID. + +Knowing were the ZCHAIN starts, helps us retrieve the ZLATEST block which we are +calling ZBLOCK. Inside of it, there is a JSON object, consisting of two keys: +1. "block" and +2. "block_signature" + +Both of these, have values that are formatted as IPFS v0 CID. "block" points to +another JSON object and "block_signature" points to a GPG armored detach +signature of BLOCK file. + +Inside BLOCK JSON object, there are the following: +1. "action", +2. "timestamp", +3. "data", +4. "data_signature", +5. "gpg" and +6. "previous" + +1. ACTION +It contains a string value. It can be written as a path but describing an action +for the object in DATA. Currently there is "news/add", "mixtapes/add" and some +more. Because of that, we can also parse this string as MODULE/ACTION. + +2. TIMESTAMP +It contains a string value. The format should be in UTC seconds as in `date -u +%s`. + +3. DATA +It contains an IPFS v0 CID of a file which is a JSON object. This also contains +the following keys: + 1. "ipfs" + 2. "detach" + 3. "timestamp" (optional) + 4. "filename" + + 1. IPFS + It contains an IPFS v0 CID of a file that we want to host in our chain. It + can be a text file or anything else. + 2. DETACH + It contains an IPFS v0 CID of a GPG armored detached signature for the IPFS + file above. + 3. TIMESTAMP + It contains a string value. The format should be in UTC seconds as in + `date -u +%s`. However, this is optional field. + 4. FILENAME + Another optional field. It can be the name of the file if we want to refer + to it with a name. + Some ACTIONS come with custom DATA JSON objects and maybe adding more keys to + the default DATA JSON object. For example, "title", "artist" and "duration". + Missing an ACTION "decoder" we just need to verify the signature of the file. + + + +4. DATA_SIGNATURE +It contains an IPFS v0 CID of a GPG armored detached signature for the DATA +file. + +5. GPG +It contains an IPFS v0 CID of a GPG armored public key. It should be the one that +signs the DATA and BLOCK. + +6. PREVIOUS +It contains an IPFS v0 CID of the previous ZBLOCK of the particular ZCHAIN we +are on. + +# Latest experiments + +- ak-sm-filesplitter +- ak-sm-filejoiner +These tools above can be used to split a file into 1024byte pieces. +`ak-sm-filesplitter` will split it and store its pieces renamed to each piece's +SHA512 hash. It also produces a text file that lists each piece's hash and a +preformatted filename, along with the original file's SHA512 hash and its +original filename. The file is called MAP +`ak-sm-filejoiner` would get this MAP and reassemble the file into your current +directory. + +Adding these two tools, came with the introduction of `ak-sm-files` script. +This script will take this map, add its hash to the "shamap" key in the DATA +JSON object, as well as its GPG armored detached signature in "shamapsig". +Currently though, the "ipfs" key has the IPFS v0 CID of the original file which +is subject to change. + +Furthermore, with the introduction of these tools, ZCHAIN could point to SHA512 +hashes that point to maps instead of IPFS v0 CIDs. + +MAP contains X+1 MAPLINES. A MAPLINE is consisting from a SHA512 hash, two +spaces and a filename up to 254 characters. This filename hosts also a piece +number. The +1 line is the SHA512 hash of the original hash, two spaces and the +original filename. + +However, this is the initial design of the `ak-sm-*` "series". + +Now we have to make the MAP just a pointer to a MerkleTree based on SHA512 hash. +We should keep that last line, but instead of the original hash, we provide the +root of the MerkleTree of that file. + +# Big picture + +Well, this is another part. Experimentaly, there are some tools introduced. +These are used in the following manner and with the following purpose. + +We introduced a SCHAIN. It consists from the following parts: +1. zblocks +2. zpairs +3. reward +4. miner +5. previous +6. nonce +7. timestamp + +After considering the above as already implemented in a way, it could be of use +to add one more +8. blocknumber + +But before adding, let's see what's already there. + + 1. ZBLOCKs + Array of IPFS v0 CIDs that point to individual ZBLOCKs. These has to be valid + ZBLOCK that is properly signed and goes back to a valid or broadly acceptable + GENESIS block. + 2. ZPAIRs + Array of JSON objects. Each object consists of 2 keys. "zchain" and "zlatest". + The "zchain" part is an IPNS key and "zlatest" is an IPFS v0 CID pointer to a + ZBLOCK, supposingly the ZLATEST of the ZCHAIN. + 3. REWARD + Reward is a supposed retribution for the contribution of the miner that made + up this SBLOCK. + 4. MINER + This is the address of the miner. It is a IPFS v0 CID pointing to a GPG + armored public key. + 5. PREVIOUS + This is an SHA512 hash value which is the hash of the previous SBLOCK. + 6. NONCE + Nonce is a random number that when is added in the forming SBLOCK would result + a file whose hash would start with some acceptable value. In our example, the + difficulty of the problem is on finding a SBLOCK whose hash starts with 3 + zeros. + 7. TIMESTAMP + As previously, `data -u +%s` date and time format. -u stands for UTC time. + + Now we can say that we would like also the eighty proposed key: + 8. BLOCK_NUMBER + This should be start counting from 1 with PREVIOUS pointing to an agreed + GENESIS block. + +# About GENESIS block of the SCHAIN + +A nice GENESIS block would be nice to consist of an outline of the whole design. +It would also be nice to host on some ZCHAIN a couple of tools as well. + +# Things that should be figured out about SCHAIN + +1. From where the REWARD is verified? + +We could have either a static REWARD or a scaling one. + +2. Limit of total REWARD amount can be in circulation? + +As of creation of ARCHINGKAOS in Stellar Network is 10000 with 7 decimal digits, +we would say that in integer value we can have as much as 100,000,000,000. That +would be the limit. + +3. How a transaction can be considered valid? + +Transactions are part of the ZCHAIN implementation. In order to make use of them +in a valid way, you should have rewards or previously transfered REWARDs from +someone that had also either won them or previously transfered to them, etc. + +Also, this has to have happened at least 6 blocks ago, just to ensure that the +SBLOCK containing the enabling transaction is final and well established. + +4. Do we have to collect fees? From where and how? + +Possibly, yes. In a bitcoin manner. We could "sell" the bytes that someone is +posting with announcing their zblock or zchain link. + +The miners of the network should validate the fee amounts, the size of the +ZBLOCK plus the size of the in between ZBLOCKS until the last one already known. + +The fees should be calculated by the contributor of the ZCHAIN. It should be +also the final ZBLOCK. + +A fee zblock could be like this: +``` +{ + "id": "1", + "zblock": "QmTkgtbZPGFyLHz1eigYQGLMiijPh3TqkAemi9gxBNZFVo", + "block": "QmaYFNVuEMnFH2fG5Czz7pWzJC1PX23d9RnyowcGgogyhY", + "timestamp": "1686961638", + "block_signature": "QmZyTrMn286zeGAZyr9ncoLKdSP5VACeQBtxQY5ksxQ2qL", + "detach": "QmZcbxbnXkCQu6aeZCu83dUxNQ6khETDxVn54TmLMDGid5", + "module": "pay", + "action": "fee", + "gpg": "QmaNjKETyxBGKVnAuV9iUkbTge5cFf4m8K1sekCeKMRAL5", + "data": "QmUtJypVap2zZPnSd7ws32kUhVmtvDVheWNKMtLV3RqiJh", + "QmUtJypVap2zZPnSd7ws32kUhVmtvDVheWNKMtLV3RqiJh": { + "fee": "100", + "ipfs": "QmVfPep9KcczZwgvufRTnz9LFFFWpEjmFNj5NrMXJxL4Me", + "detach": "QmVE4Ejb6hdErXrJmkjPjPequZJfpwQnuvUiXYJiyyPX4b" + }, + "previous": "QmeScAUMNeBYJanAuSTRo9RZi4QCdJuD8xdrS5SniygDPH" +} +``` + + +However, this means that the fee ZBLOCK alone costs 654 without even considering +the sizes of the block_signature, data_signature, ipfs and detach signature. The +other solution would be to just add a "fee" key in our BLOCK. But we can also +use the before mention method to pay for multiple ZBLOCKS. + + +5. Who holds the REWARDs and gets the FEEs as well? + +We will baptize a system as "coinbase" which would be responsible for gathering +the fees and giving out the relative rewards per mined SBLOCK to the miner. If +the coinbase is not satisfied with a proposed SBLOCK, then it gets rejected. + + +# Abstraction + +## About SBLOCK +Since almost everything is a sha512sum and the SBLOCK can have 8 keys, we could +remove the tags and set a format that would sum up to 1024 bytes. +1. MerkleTree to a list of ZBLOCKs +2. MerkleTree to a list of ZPAIRs +3. MerkleTree to the public GPG key of the MINER +4. MerkleTree of the PREVIOUSly mined SBLOCK + +Non SHA512 values: +5. REWARD amount +6. NONCE value added to mine successfully +7. TIMESTAMP in UTC +8. BLOCKNUMBER to just know where we are + +## Revisiting ZBLOCK structure + +ZBLOCK +IPFS v0 CID -> SHA512 hash of a file containing... the following + +ZBLOCK inside +``` +{ + "block": "IPFS v0 CID -> SHA512 hash", + "block_signature": "IPFS v0 CID -> SHA512 hash" +} +``` +BLOCK inside +``` +{ + "id": "1", + "timestamp": "1686961638", + "detach": "IPFS v0 CID -> SHA512 hash", + "module": "pay", + "action": "fee", + "gpg": "IPFS v0 CID -> SHA512 hash", + "data": "IPFS v0 CID -> SHA512 hash", + "previous": "IPFS v0 CID -> SHA512 hash" +} +``` +A proposal also would be to remove "action" and "module" and replace it with +the MerkleTree root of the script made the block. + +This script would address creation and handling of the "data" key and furtherly +JSON object. + +Note also that "module" is NOT declared directly in its own key, but it is +extracted from the first part of the "action" key, when key is in the form of +"module/action" format. + +By changing the "action" to splitted "module" "action", we can then use "module" +to declare our script that reads the DATA JSON object. + +Putting, though, a script that can be possibly vulnurable or malicious raises +questions. How we can implement a decoder script that we can safely execute in +our system? How we protect from eg a wallet stealer or other malware related +scripts? + +Isolated environment? + +Signed modules? +Accept signed modules from specific public GPG keys that should be declared in +the GENESIS block of the SCHAIN initially. After the initial GENESIS +declaration, initial module contributors, could sign new contributors' scripts +or people could also vote for the feature, or accept it for their node only. + +Therefore, ACTION could be performed optionally and accepted by the miners also +optionally. For example, they could make up a whitelist and/or a blacklist for +certain scripts. + +DATA inside + +``` +{ + "timestamp":"1686961638", + "fee": "100", + "ipfs": "IPFS v0 CID -> SHA512 hash MerkleTree", + "detach": "IPFS v0 CID -> SHA512 hash" +} +``` + +# Disagreements + +Talking about the ACTION scripts above, comes with a possibility of disagreement +in the network. X approves Y but W doesn't. Options: + +1. If X is part of the GENESIS block but Y isn't, then W can continue trusting +X without being obligated to run scripts provided by Y. In that sense, W can +choose to not propagate Y related ZBLOCKs. Meaning not only ZBLOCKs that are +announced to the network signed by Y but also other ZBLOCKs that contain actions +that use Y's script. + +W now can choose one of the following: +1. Fork the chain declaring their disagreement in a ZBLOCK. This should be mined +by someone or them. Next, people that agree with W should combine their powers +to continue the fork. +2. Don't fork but do not keep the relative information. Either the script or the +ZBLOCKs that are refering to it. However, if this strategy is adopted by +everyone then the SCHAIN will be not be able to be crawled back. For that could +leave an instruction to skip the particular SBLOCKs or ZBLOCKs. + +# Speaking of crawling back + +Because of the design of the ZCHAIN, the tools that got developed were to create +DATA, BLOCKs and ZBLOCKs with the purpose of getting chained together in one +ZCHAIN, which would be able to be read by crawling from the latest ZBLOCK to the +GENESIS block. The opposite direction for retrieving the ZCHAIN is not created. + +For that, in a let's say, initial state of our node, we could just ask other +nodes for the current hash of the SCHAIN. From there, we could crawl back to the +GENESIS block of the SCHAIN. This should be done first to index the SCHAIN. + +After that, we only have ( SBLOCK_AMOUNT + GENESIS ) * 1024 bytes of data and a +lot of work to do with these. If we were about to calculate that each new SBLOCK +is mined in around 1 minute, then for a year we have around 500MB for storing +just SBLOCKs. With these data, we can calculate if the amount of blocks is close +to the expected block amount, given the declarations in GENESIS block. + +``` +// Timestamps are in seconds +expected_block_number_now = floor((now_timestamp - genesis_timestamp) / 60 ) +``` + +# Other things to consider + +No idea, the train of thought reach its final destination. We will be turning it +around and be back with an even better overview and probably with some of the +above mentioned features, implemented or something. + +For sure, this document could be splitted in a way so it can fit in a +comprehensive way into proposals in the appropriate repository. + +Other than that, if you really reached this lines, a huge thank you. + +More than happy to hear any feedback from you at the IRC place: +https://irc.arching-kaos.net + |