Activation Store

However that would be better if the one server keep generate the whole layer’s activation and publish to other nodes to train each layer or through MPI/HTTP communication.