Deep learning applications are used across all life domains, and inference applications are getting deployed everywhere from data center to edge devices. This brings an appealing opportunity to create an accelerator to optimize inference flow. In this talk I will describe the software challenges we faced in building such an accelerator