URL Analysis with urlscan

Purpose and Scope

urlscan.io provides detailed analysis of web pages including screenshots, DOM content, network requests, and technology detection. This playbook covers how to use urlscan and similar tools to investigate suspicious URLs and domains safely.

Prerequisites

• urlscan account: Free account for basic scans, paid for private scans
• API access: For automation and bulk analysis
• Safe analysis environment: Do not click suspicious links directly
• URL defanging knowledge: How to safely handle malicious URLs

What urlscan Provides

Visual Analysis

• Full page screenshot of the rendered page
• DOM snapshot for content analysis
• Visible text and form fields
• Favicon and branding detection

Technical Analysis

• All HTTP requests made by the page
• JavaScript files loaded and executed
• Cookies set by the page
• Redirect chains from initial URL to final destination
• TLS certificate information

Infrastructure Information

• IP addresses and ASN information
• DNS records for the domain
• Technologies and frameworks detected
• Third party services and trackers

Analysis Workflow

1. Submit URL for Scanning

• Use the search bar or API to submit URLs
• Choose visibility: public, unlisted, or private (paid)
• Select user agent and location if needed
• Wait for scan to complete (usually 30 to 60 seconds)

2. Review Screenshot

Start with visual analysis:

• Does the page impersonate a legitimate brand?
• Are there login forms or payment requests?
• Does content match the claimed purpose?
• Are there obvious red flags (poor design, errors)?

3. Examine Redirect Chain

• Track the path from initial URL to final destination
• Identify intermediate redirectors or shorteners
• Note any cloaking or conditional redirects
• Document all domains in the chain

4. Analyze Network Requests

• What external domains are contacted?
• Are any requests to known malicious infrastructure?
• Is data being sent to unexpected destinations?
• Are there downloads or script injections?

5. Review Technologies

• What CMS or framework is used?
• Are there known vulnerable components?
• Is the hosting typical for legitimate sites?
• Does the technology match the claimed organization?

6. Check Historical Data

• Has this URL or domain been scanned before?
• How has the content changed over time?
• When was the page first seen?
• Are there related scans for similar pages?

Identifying Phishing Pages

Visual Indicators

• Brand logos and styling copied from legitimate sites
• Login forms asking for credentials
• Urgency messaging (account suspended, verify now)
• Poor quality images or formatting errors

Technical Indicators

• Recently registered domain
• Free hosting or subdomain services
• Form submissions to different domains
• Obfuscated JavaScript
• Data exfiltration to third party servers

Infrastructure Indicators

• Hosting in unexpected geography
• Shared hosting with other suspicious sites
• Recently issued SSL certificates
• Domain name typosquatting or lookalikes

Similar Tools

Hybrid Analysis

• Sandbox execution of URLs and files
• Network traffic capture
• Behavioral analysis

Any.run

• Interactive sandbox for URLs
• Real time analysis and control
• Process and network visualization

Google Safe Browsing

• Check if URL is on Google's blocklist
• API available for bulk checking
• Integrated into Chrome and other browsers

API Integration

• Submit scans programmatically
• Search historical scan data
• Retrieve results in JSON format
• Integrate into SOAR playbooks for auto analysis

Privacy and Visibility

• Public scans: Visible to everyone, indexed by search
• Unlisted scans: Not indexed but accessible via direct link
• Private scans: Only visible to you (paid feature)
• Be cautious scanning internal URLs on public visibility

Escalation Guidance

Escalate when analysis reveals:

• Active phishing page targeting your organization
• Credential harvesting forms
• Malware distribution pages
• Pages impersonating executives or partners
• Evidence of user interaction with malicious page

References

• urlscan.io: urlscan.io
• urlscan API documentation: urlscan.io/docs/api
• Google Safe Browsing: safebrowsing.google.com
• Hybrid Analysis: hybrid-analysis.com
• Any.run: any.run