How To Find Element By Text Using Selenium

Finding a web element by its visible text is one of the most common tasks in browser automation and web scraping. Selenium offers several strategies to do this cleanly, and choosing the right one depends on whether you need an exact match, a partial match, or something more flexible. Understanding the tradeoffs between XPath expressions and other locator strategies will make your automation scripts more reliable and easier to maintain.

This reference covers the most practical approaches for locating elements by text in Selenium using Python, along with tips for handling dynamic pages, avoiding brittle selectors, and connecting your scripts to a proxy layer when you need to run large-scale data collection without interruption.

Why Text-Based Locators Are Useful

HTML attributes like id and class are ideal locators, but real-world pages do not always provide stable, unique identifiers on every element. Visible text — button labels, link text, table cell values — tends to be more stable from a functional standpoint, since changing it would visibly break the UI. For developers doing python web scraping or automating form interactions, text-based locators are a practical fallback when structural attributes are absent or inconsistent.

That said, text-based locators can break when content is localized, dynamically loaded, or trimmed with whitespace. It helps to understand a few different strategies so you can pick the most appropriate one for your situation.

Using link_text and partial_link_text

Selenium's built-in link_text and partial_link_text locators work specifically on anchor (<a>) elements. They are the simplest approach when you need to click a hyperlink whose label you know.

• link_text — matches the full, exact visible text of the link. Case-sensitive and whitespace-sensitive.
• partial_link_text — matches any link whose visible text contains the given substring. Useful when the full label may vary slightly.

Example in Python:

driver.find_element(By.LINK_TEXT, "Sign In")
driver.find_element(By.PARTIAL_LINK_TEXT, "Sign")

These methods only work on <a> tags. For buttons, spans, divs, or other elements, you need XPath.

Using XPath with text() and contains()

XPath is the most versatile approach for finding any element by its text content. Two XPath functions are particularly relevant here.

Exact match with text():

driver.find_element(By.XPATH, '//*[text()="Submit"]')

This finds any element whose complete text node equals "Submit". It is case-sensitive and will not match if there is leading or trailing whitespace around the text in the DOM.

Partial match with contains():

driver.find_element(By.XPATH, '//*[contains(text(), "Submit")]')

This is more forgiving and will match elements whose text node includes the substring. It is particularly useful when the full text includes dynamic values, counters, or extra whitespace you cannot fully predict.

To narrow matches to a specific element type — which reduces the chance of selecting an unintended element — prefix with the tag name:

driver.find_element(By.XPATH, '//button[contains(text(), "Submit")]')

Handling normalize-space for Whitespace Issues

One of the most common reasons a text-based XPath fails silently is invisible whitespace: newlines, tabs, and extra spaces that appear in the raw HTML but not on screen. The XPath normalize-space() function collapses all internal whitespace and trims leading and trailing space, making matches much more reliable on real-world pages.

driver.find_element(By.XPATH, '//*[normalize-space(text())="Add to Cart"]')

This technique is especially valuable when scraping e-commerce or data-heavy pages where developers may not have been consistent about whitespace in their markup.

Combining Text Matching with Other Attributes

When text alone is not unique enough — for example, a page with multiple "Edit" buttons — you can combine text conditions with other attribute predicates in the same XPath expression:

driver.find_element(By.XPATH, '//button[@class="primary" and contains(text(), "Edit")]')

You can also use the ancestor or parent axis to scope a text search within a particular section of the DOM, which is helpful during web scraping when a page repeats the same labels in different containers.

Connecting Selenium to a Proxy for Scraping at Scale

Once your element-locating logic is solid, the next challenge for large-scale python web scraping projects is avoiding blocks and rate limits. Websites detect repeated automated requests by IP address and may return CAPTCHAs, throttled responses, or outright bans — making all your carefully crafted selectors useless.

Routing your Selenium sessions through rotating proxies is the standard solution. You configure a proxy in Selenium's options before launching the browser:

options.add_argument('--proxy-server=http://your.proxy.host:port')

For production scraping work, residential or rotating datacenter proxies tend to be more effective than a single static IP. Services that offer rotating pools are worth evaluating carefully for reliability and session persistence. Cheapest Proxies is worth considering for buyers comparing affordable proxy services who want rotating options without a large upfront commitment. When evaluating any provider, look at whether they support session stickiness, their rotation interval, and how they handle authentication.

Using proxies for scraping alongside well-structured Selenium selectors gives you both the technical precision to extract the right data and the infrastructure to do it at scale.

Best Practices and Common Pitfalls

• Always prefer a stable id or data-* attribute over text matching when one is available — text is readable but can be translated or reworded.
• Use explicit waits (WebDriverWait with expected_conditions) rather than implicit waits or time.sleep() when waiting for text to appear dynamically.
• Test your XPath in the browser DevTools console using $x('//button[contains(text(),"Submit")]') before putting it in code.
• Be aware that text() only matches direct text nodes, not text inside child elements. Use string() or .//text() if the text is split across nested tags.
• Wrap text-based locators in try/except blocks and log failures clearly — text changes are a common source of hard-to-debug breakage in long-running scraping pipelines.