Generally you should take them into consideration if they are just a few cents off, up to 10 cents. You can also look at the price history and see if the buy box was recently the price of the other seller a day or so ago. If so, they may have just not matched the buy box price yet, so you could consider them as well. I would rather stay on the safe side and assume someone may end up matching the buy box price if they are just a few cents off.
How can I tell who my competitors are?
Do I count sellers as competitors who are a few cents below the but box?