That's correct, unless of course your character has a Combat Tactic such as Lethal Blow, or Brutal that makes th ball into an actual weapon. (I would think that they are considered unarmed combat since they are weapon level 0)
Of course, once we start enchanting the balls with spells, then whatever weapon level the caster can make the ball, the ball takes on that property.