reference_guide.md

📘 deeplearn_numpy Reference Guide

NumPy-only deep learning mini-framework – single-file, extensible, and educational.

---

🔹 Core Classes

Parameter

p = Parameter(data: np.ndarray, name: Optional[str] = None)

• .data → weights (NumPy array)
• .grad → gradients
• .zero_grad() → reset gradients

---

🔹 Layers

All layers subclass Layer:

class Layer:
    def build(input_shape) -> output_shape
    def forward(x) -> ndarray
    def backward(dy) -> ndarray
    def params() -> Iterable[Parameter]

Available Layers

• Dense(units, use_bias=True, kernel_initializer="glorot_uniform")
• Conv2D(filters, kernel_size, stride=1, padding="valid", dilation=1, use_bias=True)
• MaxPool2D(pool_size=2, stride=None, padding="valid")
• AvgPool2D(pool_size=2, stride=None, padding="valid")
• Flatten()
• Dropout(rate=0.5)
• BatchNorm(momentum=0.9, eps=1e-5, affine=True)

---

🔹 Activations

Registered as both layers and activations:

• ReLU()
• Sigmoid()
• Tanh()
• LeakyReLU(alpha=0.01)
• Softmax(axis=-1)

---

🔹 Losses

Base:

class Loss:
    def forward(y_pred, y_true) -> float
    def backward() -> ndarray

Available:

• MSELoss
• CrossEntropyLoss(axis=-1, from_logits=True)
• BinaryCrossEntropy

---

🔹 Optimizers

Base:

class Optimizer:
    def step(params)
    def zero_grad(params)

Available:

• SGD(lr=1e-2, momentum=0.0, nesterov=False, weight_decay=0.0)
• Adam(lr=1e-3, beta1=0.9, beta2=0.999, eps=1e-8, weight_decay=0.0)

---

🔹 Metrics

accuracy(y_pred, y_true)
binary_accuracy(y_pred, y_true, threshold=0.5)
top_k_accuracy(y_pred, y_true, k=5)

---

🔹 Model

class Model:
    def __init__(layers, loss="cross_entropy", optimizer="Adam",
                 metrics=None, regularization=None, seed=None)

    def build(input_shape)
    def fit(x, y, epochs=1, batch_size=32, shuffle=True, verbose=1, callbacks=None)
    def evaluate(x, y, batch_size=128) -> Dict[str, float]
    def predict(x, batch_size=128) -> ndarray
    def save_weights(path)
    def load_weights(path)

---

🔹 Config Builder

model = build_model_from_config(config: Dict)

Example:

config = {
  "input_shape": [64, 20],
  "layers": [
    {"type": "Dense", "units": 32, "activation": "relu"},
    {"type": "Dropout", "rate": 0.1},
    {"type": "Dense", "units": 5}
  ],
  "loss": "cross_entropy",
  "optimizer": {"type": "Adam", "lr": 1e-2},
  "metrics": ["accuracy"],
  "regularization": {"l2": 1e-4}
}

---

🔹 Extensibility

Register a custom layer

@register_layer("MyLayer")
class MyLayer(Layer):
    ...

Register a custom loss

@register_loss("my_loss")
class MyLoss(Loss):
    ...

Register a custom optimizer

@register_optimizer("MyOpt")
class MyOpt(Optimizer):
    ...