Medical robots provide enhanced dexterity, vision, and safety for a broad range of procedures. In this article, we present a handheld, robotic device capable of performing peripheral catheter insertions with high accuracy and repeatability. The device utilizes a combination of ultrasound imaging, miniaturized robotics, and machine learning to safely and efficiently introduce a catheter sheath into a peripheral blood vessel. Here, we present the mechanical design and experimental validation of the device, known as VeniBot. Additionally, we present results on our ultrasound deep learning algorithm for vessel segmentation, and performance on tissue-mimicking phantom models that simulate difficult peripheral catheter placement. Overall, the device achieved first-attempt success rates of 97 ± 4% for vessel punctures and 89 ± 7% for sheath cannulations on the tissue mimicking models (n = 240). The results from these studies demonstrate the viability of a handheld device for performing semi-automated peripheral catheterization. In the future, the use of this device has the potential to improve clinical workflow and reduce patient discomfort by assuring a safe and efficient procedure.