Artificial Intelligence

A bit of a hobby of mine is Artificial Intelligence. I have studied this for some time now, and I have a half Reasonable grasp on much of this area.

The Transformer Model Architecture, from the Paper: "Attention is all you need" ( Circa 2017 ) is a Ground Breaking Leap Forward in the AI Arena! Of course, this is the origins behind Chat GPT ( Generative Pre-Trained Transformer. )

I use CNTK, because its a C# Compatible AI Architecture and its what I know, even if it is somewhat old and out of date. I hope Microsoft does bring back Support, because I cant stand ML.NET, its just awful!

March of 2016 I completed an Stanford Machine Learning Qualification, giving me a small head start in the AI Field, but it took me some time to develop the necessary skill to make AI useful in everyday life!

Have I built a Transformer, yes, here is a small one, a baby, a Single Layer GPT:

        public async void GPTAsync(int Heads = 8)
        {

            // Load Vocabulary:
            if (await LoadVocabularyAsync())
            {

                // The Vocab Size:
                int VocabSize = WordToIndex.Count;

                // Mask Variables:
                float Multiplier = 1.0f;
                float TrilMask = float.NegativeInfinity;

                // The Input Variables:
                CNTK.Variable DecoderInputs = Variable.InputVariable(new int[] { DecoderRowDimension }, DataType.Float, ModelInputName);

                // Init a new Mask:
                CNTK.Constant Mask = Build.Mask(DecoderRowDimension, Multiplier, TrilMask);

                // The Encoder / Decoder Blocks: Nx times
                CNTK.Function LinearDecoderBlock = new LinearDecoderBlock(DecoderInputs, Mask, Heads, DenseLayerDimension, DecoderRowDimension).Output;

                // Send through A Fully Connected Feed-Forward Network.
                Function DenseLayer1 = Layers.DenseLayer(LinearDecoderBlock, DenseLayerDimension, Activation.ReLU, "InputsX");
                Function DenseLayer2 = Layers.DenseLayer(DenseLayer1, DenseLayerDimension, Activation.ReLU, "LayerOne");
                Function DenseLayer3 = Layers.DenseLayer(DenseLayer2, VocabSize, Activation.ReLU, "OutputsY");

                // Adding Dropout and run a Softmax to get Output Probabilities:
                Model = CNTK.CNTKLib.Dropout(CNTKLib.Softmax(DenseLayer3), 0.1);
            }
        }

This small prototype shows how simple a basic Model can look, but there is more complexity under the hood!

Being able to apply Artificial Intelligence to some medial tasks does help in a lot of areas, especially when one person has no time to get many things done!