Better Building Blocks for your Neural Networks
The GeoND library is a collection of custom ops for PyTorch and TensorFlow, that are model/data agnostic and provide improved performance.
Key points
More powerful neurons.
Unlike traditional neurons with linear decision boundaries, the GeoND library provides layers with Paraboloid neurons, a type of neuron whose decision boundary is a 2nd degree curve. Paraboloid neurons do not require a full covariance matrix and only involve double the parameters of traditional neurons. The library is GPU-enabled and optimized to provide competitive runtimes.
Improved performance.
Replacing an existing layer with a layer of the same number of paraboloid neurons, or adding new paraboloid layers and following the same training process can result in improved performance.
Faster inference.
Alternatively, since paraboloid neurons are more powerful, replacing a layer of traditional neurons with a layer of fewer paraboloid neurons can result in a net gain in execution speed.
Easy to adopt.
The GeoND library modules are designed to receive the same arguments with the already existing PyTorch and TensorFlow modules and their parameters are tuned to work out of the box.
Access to independent research.
The GeoND library is the result of on-going independent research into the building blocks of neural networks that aims to continue adding new and better custom ops.
Usage examples
(Ready to release)
(Coming soon)
import torch
import torch.nn as nn
import torch.optim as optim
import geondpt as gd
Install the appropriate package using pip and then import it to your code.
#coming soon
#self.conv1=nn.Conv2d
self.conv1=gd.ParaConv2d
(in_channels,
out_channels,
kernel_size=3
stride=stride,
padding=1)
Replace a traditional layer with a paraboloid layer. In this case, it's a convolutional layer being replaced.
#coming soon
#self.out=nn.Linear(n_in,
# n_classes)
self.out=nn.Sequential(
gd.Paraboloid(n_in, 1024)
nn.Linear(1024, n_classes)
)
Insert a paraboloid layer before the output layer. Avoid using a paraboloid layer as the output layer.
#coming soon
#optimizer = optim.SGD
optimizer = gd.GeoNDSGD
(net.parameters(),
lr=0.001,
momentum=0.9)
Remember to use the GeoND optimizer, which properly handles paraboloid layers.
#coming soon