It is possible you had a stress fracture. They do not always show up on an X ray immediately. Certainly a bone scan will be more helpful. If it shows uptake in the 3rd phase, you probably have a fracture.
I agree with immobilizing the foot if there is an injury. Why didn't you tell your doctor your foot continued to hurt in the cast? You might need further immobiliztion or even physical therapy. Talk to your doctor.