If you do not give PoA to anyone, you are the sole operator of your Demat account. If you sell the share today, you have to transfer the share to the broker's Demat account by the next working day so that he can deliver them to the buyer. If you fail to do that, it is the broker's obligation to provide the shares to the buyer. If can the share doesn't reach the broker in time, he has to buy them from the market and settle the transaction. This may result in penalties for you. You also have to pay the difference in the prices.
To avoid this manual transaction, almost every investor gives POA, for convenience in trade settlements.