Sur.ly: Developers Guide

We aim to create versions of Sur.ly plugin for every popular CMS and forum management software available on the market, however, we cannot yet cover all existing systems. In fact, it’s quite easy to integrate Sur.ly in any website, whether it is powered by a generic CMS or a unique custom solution.

  • Sur.ly PHP SDK developers guide

    Quick start

    To let Sur.ly protect your website and its visitors from potential threats coming from suspicious links in user-generated content, you just need to add Sur.ly prefix to that links. Like this:

    
    http://theverge.com => http://sur.ly/theverge.com
    http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/ => http://sur.ly/techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/
    

    You can remove “http://”, “https://” and “www.” prefixes. We recommend that you encode links according to RFC 3986.

    Full-featured integration

    To allow full-featured Sur.ly integration, including inbuilt links shortening service, we developed Sur.ly Software Development Kit. This kit uses PHP 4.3 or greater. It’s distributed according to LGPL license and is easy-to-use

    Download

    Toolbar ID

    Start with creating your account on Sur.ly and adding your website there. You’ll get Toolbar ID in the following format: AA000000, and will be able to access toolbar visual settings. Users will see the toolbar at the top of the target page when they follow an outbound link replaced by Sur.ly.

    Installation of Sur.ly SDK

    To enable link replacement you need to create an instance of Surly PHP class and use one of the provided methods:

    • process() processes links in HTML code, for example the whole post or user comments;
    • processUrl() processes a specific link;
    • processMultipleUrls() processes an array of links. Links in the resulting array will be in the same order as they were at the input.

    Usage example:

    
    $toolbarId = "AA000014";
    
    require_once('/path/to/surly-sdk-php/Surly.php');
    $surly = new Surly($toolbarId);
    
    $customHtml = $surly->process($html);
    // $customHtml now contains links protected by Sur.ly
    
    $customLink = $surly->processUrl("http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/");
    // $customLink is now http://sur.ly/o/techcrunch.com/2014%2F06%2F22%2Fheartbleed-isnt-dead-yet/AA000014
    
    $linkArray = array(
    "http://techcrunch.com",
    "http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/",
    );
    $customLinkArray = $surly->processMultipleUrls($linkArray);
    // $customLinkArray now contains:
    // http://sur.ly/o/techcrunch.com/AA000014
    // http://sur.ly/o/techcrunch.com/2014%2F06%2F22%2Fheartbleed-isnt-dead-yet/AA000014
    

    Disabling inbound links replacement

    To disable replacement of inbound or outbound links to trusted domains (for example google.com) you need to add such domains into white list:

    
    $surly->whitelist(@$_SERVER['HTTP_HOST']);
    $surly->whitelist("google.com");
    
    $customLink = $surly->processUrl("http://google.com");
    // $customLink is still http://google.com
    

    Link shortening

    Sur.ly provides links shortening service that works to hide any unwanted text in links and make them look neat and convinient. Like this:

    
    http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/ => http://sur.ly/tUw
    

    To enable links shortening you need to pass true value for the second parameter with Surly class initialization:

    
    require_once('/path/to/surly-sdk-php/Surly.php');
    
    $surly = new Surly($toolbarId, true);
    
    $customLink = $surly->processUrl("http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/");
    // $customLink is now http://sur.ly/o/tUw/AA000014
    

    Attention! To shorten multiple links at a time, a sole HTTP request will be sent from your website to api.surdotly.com server. That might put additional workloads on the server and hence cause an essential slowdown of your website. To avoid such an issue, you need to set caching mechanism for shortened links on your side. For this purpose add your own class that inherits Surly class, and implement the following methods:

    • cacheShortIds() - it works for saving shortened links in cache;
    • getCachedShortIds() - it works for retrieving saved links from cache.

    This PHP 5 example demonstrates how to use a popular in-memory key-value store “Memcached” for caching tandems target URL => shorten ID:

    
    require_once('/path/to/surly-sdk-php/Surly.php');
    
    class MemcacheSurly extends Surly
    {
        function __construct(Memcache $memcache, $toolbarId)
        {
            $this->memcache = $memcache;
            parent::Surly($toolbarId, true);
        }
    
        function cacheShortIds($url2shortIds)
        {
            foreach ($url2shortIds as $url => $shortId) {
                $this->memcache->set(md5($url), $shortId);
            }
        }
    
        function getCachedShortIds($urls)
        {
            $url2shortIds = array();
    
            foreach ($urls as $url) {
                $shortId = $this->memcache->get(md5($url));
                if ($shortId) {
                    $url2shortIds[$url] = $shortId;
                }
            }
    
            return $url2shortIds;
        }
    }
    

    Download

  • Sur.ly Javascript SDK developers guide

    Sur.ly Javascript SDK developers guide

    Just copy this HTML code snippet and paste it before the closing </body> tag on your website. Once installed, Sur.ly will immediately help you take every outbound link on the page under control.

    
    <script type="text/javascript" src="//cdn.surdotly.com/js/Surly.min.js"></script>
    <script type="text/javascript">
        var surly = new Surly('AA000014');
        surly.whiteList(window.location.hostname);
        surly.process(document.getElementsByTagName('BODY'));
    </script>
    

    Toolbar ID

    Start with creating your account on Sur.ly and adding your website there. You’ll get Toolbar ID in the following format: AA000000, and will be able to access toolbar visual settings. Users will see the toolbar at the top of the target page when they follow an outbound link replaced by Sur.ly.

    Installation of Sur.ly SDK

    To enable link replacement you need to create an instance of Surly object and use one of the provided methods:

    • process() processes links inside of HTML DOM element, for example the in < ul> or < div> or anything else;
    • processUrl() processes a specific link;
    • processMultipleUrls() processes an array of links. Links in the resulting array will be in the same order as they were at the input.

    Usage example:

    
    <script type="text/javascript" src="//cdn.surdotly.com/js/Surly.min.js"></script>
    
    <div id="links-process">
        <h3>Links:</h3>
        <ul>
            <li><a href="http://cnn.com">Link</a></li>
            <li><a href="http://youtube.com">Link</a></li>
            <li><a href="http://yahoo.com">Link</a></li>
        </ul>
    </div>
    <div id="single-url">
        <h3>Single URL</h3>
    </div>
    
    <script type="text/javascript">
        // process
        var surly = new Surly('AA000014');
        surly.process(document.getElementById('links-process'));
        // resulting HTML
        //	<ul>
        // <li><a href="http://sur.ly/o/cnn.com/AA000014">Link</a></li>
        //     <li><a href="http://sur.ly/o/youtube.com/AA000014">Link</a></li>
        //   <li><a href="http://sur.ly/o/yahoo.com/AA000014">Link</a></li>
        //   </ul>
    
        // process URL
        var singleUrlContainer = document.getElementById('single-url');
        var singleUrl = document.createElement('a');
        singleUrl.setAttribute('href', surly.processUrl('http://yahoo.com'));
        singleUrl.appendChild(document.createTextNode('http://yahoo.com'));
        singleUrlContainer.appendChild(singleUrl);
    
    
        //<a href="http://sur.ly/o/yahoo.com/AA000014">http://yahoo.com</a>
    
    
        // process multiple urls
        var links = surly.processMultipleUrls(['http://cnn.com', 'http://example.com']);
        var list = document.getElementById('links-process-multiple-urls');
        var ul = document.createElement('UL');
    
        for(var i=0; i<links.length; i++) {
            var li = document.createElement('LI');
            var a = document.createElement('A');
            a.setAttribute('href', links[i]);
            a.appendChild(document.createTextNode(links[i]));
    
            li.appendChild(a);
            ul.appendChild(li);
        }
    
        list.appendChild(ul);
    
        //<ul>
        //<li>
        //<a href="http://sur.ly/o/cnn.com/AA000014">http://sur.ly/o/cnn.com/AA000014</a>
        //</li>
        //<li>
        //<a href="http://sur.ly/o/example.com/AA000014">http://sur.ly/o/example.com/AA000014</a>
        //</li>
        //</ul>
    </script>
    

    Disabling inbound links replacement

    To disable replacement of inbound or outbound links to trusted domains (for example google.com) you need to add such domains into white list:

    
    surly.whiteList(window.location.hostname)
    .whiteList('http://google.com')
    .whiteList('http://wikipedia.org');
    // the link is still http://google.com
    

    Download

  • Sur.ly Python 2 SDK developers guide

    Installation

    To enable link replacement you need to create an instance of Surly Python class and use one of the provided methods:

    • process() processes links in HTML code, for example the whole post or user comments;
    • processUrl() processes a specific link;
    • processMultipleUrls() processes an array of links. Links in the resulting array will be in the same order as they were at the input.

    Download surly-sdk-python2 package and place it in your project.

    Download

    Configuration

    You have to set up toolbar ID and your site domain name:

    
    from surly import Surly
    
    surly = Surly(
            "AA000014",  # toolbar given to you when configuring your site on sur.ly
            host=”mysite.com” # your site domain (optional) needed for sur.ly service
    )
    

    Usage examples

    HTML processing

    
    custom_html = surly.process(html)
    # custom_html now contains links protected by Sur.ly
    custom_html = surly.process(html, encoding=’cp1251’)
    # for non unicode strings you have to define encoding, otherwise surly will try to decode string itself. custom_html now contains links protected by Sur.ly
    

    Single URL processing

    
    custom_link = surly.process_url("http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/")
    # custom_link is now http://sur.ly/o/techcrunch.com/2014%2F06%2F22%2Fheartbleed-isnt-dead-yet/AA000014
    

    Multiple URLs processing

    
    link_list = [
        "http://techcrunch.com",
        "http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/",
    ]
    
    custom_link_list = surly.process_multiple_urls(link_list)
    # custom_link_list now contains:
    # http://sur.ly/o/techcrunch.com/AA000014
    # http://sur.ly/o/techcrunch.com/2014%2F06%2F22%2Fheartbleed-isnt-dead-yet/AA000014
    

    Disabling inbound links replacement

    To disable replacement of inbound or outbound links to trusted domains (for example google.com) you need to add such domains into white list:

    
    surly.add_to_whitelist(‘you_server_http_host’)
    surly.add_to_whitelist("google.com")
    
    custom_link = surly.process_url("http://google.com")
    # custom_link is still http://google.com
    

    Link shortening

    Sur.ly provides links shortening service that works to hide any unwanted text in links and make them look neat and convinient. Like this:

    
    http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/ => http://sur.ly/tUw
    

    To enable links shortening you need to pass true value for the use_shortener parameter with Surly class initialization:

    
    from surly import Surly
    surly = Surly(toolbar_id, use_shortener=True, host=”mysite.com”)
    
    custom_link = surly.process_url("http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/")
    # custom_link is now http://sur.ly/o/tUw/AA000014
    

    Attention! To shorten multiple links at a time, a sole HTTP request will be sent from your website to api.surdotly.com server. That might put additional workloads on the server and hence cause an essential slowdown of your website. To avoid such an issue, you need to set caching mechanism for shortened links on your side. For this purpose add your own class that inherits Surly class, and implement the following methods:

    • _cache_short_ids() works for saving shortened links in cache;
    • _get_cached_short_ids() works for retrieving saved links from cache.

    This Python 2 example demonstrates how to use a popular in-memory key-value store “Memcached” for caching tandems target URL => shorten ID using pylibmc lib for memccached:

    
    import pylibmc
    import hashlib
    from surly import Surly
    
    class MemcacheSurly(Surly):
    
        ROOT_STATUS_KEY = '_surly_root_status'
    
        def __init__(self, memcache_client, toolbar_id=None, use_shortener=False, host=None):
            self.memcache_client = memcache_client
            super(MemcacheSurly, self).__init__(toolbar_id, use_shortener, host)
    
        def _cache_short_ids(self, remote_short_ids):
            for url, shortened in remote_short_ids.items():
                self.memcache_client[hashlib.md5(url).hexdigest()] = shortened
    
        def _get_cached_short_ids(self, urls):
            url2shortened = {}
            for url in urls:
                try:
                    url2shortened[url] = self.memcache_client[hashlib.md5(url).hexdigest()]
                except KeyError:
                    continue
            return url2shortened
    
    my_memcache_surly = MemcacheSurly(
        pylibmc.Client(["127.0.0.1"], binary=True),
        host=”mysite.com”,
        use_shortener=True,
        toolbar_id="AA000014"
    )
    

    Download

  • Sur.ly Python 3 SDK developers guide

    Installation

    To enable link replacement you need to create an instance of Surly Python class and use one of the provided methods:

    • process() processes links in HTML code, for example the whole post or user comments;
    • processUrl() processes a specific link;
    • processMultipleUrls() processes an array of links. Links in the resulting array will be in the same order as they were at the input.

    Download surly-sdk-python3 package and place it in your project.

    Download

    Configuration

    You have to set up toolbar ID and your site domain name:

    
    from surly import Surly
    
    surly = Surly(
        "AA000014",  # toolbar given to you when configuring your site on sur.ly
        host=”mysite.com” # your site domain (optional) needed for sur.ly service
    )
    

    Usage examples

    HTML processing

    
    custom_html = surly.process(html)
    # custom_html now contains links protected by Sur.ly
    

    Single URL processing

    
    custom_link = surly.process_url("http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/")
    # custom_link is now http://sur.ly/o/techcrunch.com/2014%2F06%2F22%2Fheartbleed-isnt-dead-yet/AA000014
    

    Multiple URLs processing

    
    link_list = [
        "http://techcrunch.com",
        "http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/",
    ]
    
    custom_link_list = surly.process_multiple_urls(link_list)
    # custom_link_list now contains:
    # http://sur.ly/o/techcrunch.com/AA000014
    # http://sur.ly/o/techcrunch.com/2014%2F06%2F22%2Fheartbleed-isnt-dead-yet/AA000014
    

    Disabling inbound links replacement

    To disable replacement of inbound or outbound links to trusted domains (for example google.com) you need to add such domains into white list:

    
    surly.add_to_whitelist(‘you_server_http_host’)
    surly.add_to_whitelist("google.com")
    
    custom_link = surly.process_url("http://google.com")
    # custom_link is still http://google.com
    

    Link shortening

    Sur.ly provides links shortening service that works to hide any unwanted text in links and make them look neat and convinient. Like this:

    
    http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/ => http://sur.ly/tUw
    

    To enable links shortening you need to pass true value for the use_shortener parameter with Surly class initialization:

    
    from surly import Surly
    surly = Surly(toolbar_id, use_shortener=True, host=”mysite.com”)
    
    custom_link = surly.process_url("http://techcrunch.com/2014/06/22/heartbleed-isnt-dead-yet/")
    # custom_link is now http://sur.ly/o/tUw/AA000014
    

    Attention! To shorten multiple links at a time, a sole HTTP request will be sent from your website to api.surdotly.com server. That might put additional workloads on the server and hence cause an essential slowdown of your website. To avoid such an issue, you need to set caching mechanism for shortened links on your side. For this purpose add your own class that inherits Surly class, and implement the following methods:

    • _cache_short_ids() works for saving shortened links in cache;
    • _get_cached_short_ids() works for retrieving saved links from cache.

    This Python 3 example demonstrates how to use a popular in-memory key-value store “Memcached” for caching tandems target URL => shorten ID using pylibmc lib for memccached:

    
    import memcache
    from surly import Surly
    import hashlib
    
    
    class MemcacheSurly(Surly):
    
        ROOT_STATUS_KEY = '_surly_root_status'
    
        def __init__(self, memcache_client, toolbar_id=None, use_shortener=False, host=None):
            self.memcache_client = memcache_client
            super(MemcacheSurly, self).__init__(toolbar_id, use_shortener, host)
    
        def _cache_short_ids(self, remote_short_ids):
            for url, shortened in remote_short_ids.items():
                self.memcache_client.set(hashlib.md5(url.encode()).hexdigest(), shortened)
    
        def _get_cached_short_ids(self, urls):
            url2shortened = {}
            for url in urls:
                url_shortened = self.memcache_client.get(hashlib.md5(url.encode()).hexdigest())
                if url_shortened is None:
                    continue
                url2shortened[url] = url_shortened
    
            return url2shortened
    
    
    my_memcache_surly = MemcacheSurly(
        memcache.Client(["127.0.0.1:11211"], debug=0),
        host="myhost.example.com",
        use_shortener=True,
        toolbar_id="MyToolBarId"
    )
    

    Download

  • Sur.ly Django SDK developers guide

    Installation

    To enable link replacement you need to create an instance of Surly Python class and use one of the provided methods:

    • Download django-surly and install it inside your django project:
      • For system python:
        pip install --user django-surly-1.0.tar.gz
      • For virtualenv:
        pip install django-surly-1.0.tar.gz
    • Add "surly" to your INSTALLED_APPS in settings.py like this:
      
      INSTALLED_APPS = (
          ...
          'surly',
      )
      

    Configuration

    In project settings.py you should:

    • set up your surly toolbar ID:
      SURLY_TOOLBAR_ID = 'your-toolbar-id-from-sur.ly'
    • set up surly remote shortener. It gives an opportunity to use sur.ly shortener for URLs to be processed remotely. In this case your URLs will be processed in the most short way, otherwise all your URLs will be processed regularly:
      SURLY_ENABLE_SHORTENER = True
    • set up domain that you do NOT want to be processed at all:
      SURLY_WHITE_LIST_DOMAINS = ('google.com', 'mydomain.com')
    • finally set up your host (domain):
      SURLY_HOST = "mydomain.com"

    Usage examples

    Surly filter template feature

    
    {% load surly %}
    {{ '<a href="http://google.com">Google.com</a>'|surly }}
    

    Surly tag template feature

    
    {% load surly %}
    {% surly %}
        <a href="http://google.com">Google.com</a>
    {% endsurly %}
    

    Surly middleware

    Open project settings.py and add surly.middleware.SurlyMiddleware to MIDDLEWARE_CLASSES.

    WARNING! Be sure to use template features OR middleware, but not both. Otherwise your urls will be processed twice.

    • set it in settings:
      
      MIDDLEWARE_CLASSES = (
          ...
          'surly.middleware.SurlyMiddleware'
      )
      
    • or set it in a view method as decorator:
      
      from django.utils.decorators import decorator_from_middleware
      from django.views.generic import TemplateView
      from surly.middleware import SurlyMiddleware
      
      class MyAppView(TemplateView):
          template_name = "index.html"
      
          @decorator_from_middleware(SurlyMiddleware)
          def get(self, request, *args, **kwargs):
              return super(self.__class__, self).get(request, *args, **kwargs)
      

    Now reload the server and use sur.ly shortener!

    Uninstall

    To uninstall the surly application simply remove surly from INSTALLED_APPS in settings.py and uninstall the app itself with the following command:

    pip uninstall django-surly

    Download

  • Sur.ly ASP.NET SDK developers guide

    Installation

    There are two ways of Surly ASP.NET configuration: using web.config and directly calling Surly lib methods

    Configuration using web.config

    Download Surly ASP.NET module and place it to your project. Here are the instructions for configuration:

    • In case you need to process all outgoing pages just register Surly ASP.NET module in configuration section:
      
      <system.webServer>
          <modules>
              <add name="SurlyModule" type="Core.SurlyModule"/>
          </modules>
      </system.webServer>
      
      And then enable processing in appSettings section:
      <add key="Surly:htmlParsingEnabled" value="true"/>
      Finally add your Toolbar ID to the project configuration in appSettings section:
      <add key="Surly:toolBarId" value="AA000014"/>
    • Optionally, it is possible to turn URL shortener on just adding the following line in appSettings section:
      <add key="Surly:useShortUrls" value="true"/>
    • If you don’t need to process specific or trusted URLs or domains, just exclude them in appSettings section as follows using “;” as a delimiter for values:
      <add key="Surly:whiteListUrls" value="bing.com;yahoo.com;google.com"/>

    Configuration direct Surly ASP.NET module usage

    It’s also easy to programmatically configure Surly ASP.NET module for special needs. There are several tools for the developer to process URLs as efficient as needed:

    • If you need to process specific URL. You need to pass an URL and a toolbar ID given on the administration panel on Sur.ly:
      
      public static string ProcessUrl(string link, string toolBarId)
      // http://techcrunch.com => http://sur.ly/o/techcrunch.com/AA000014
      
    • Processing multiple URLs. You need to pass a dictionary of links:
      public static Dictionary<string, string> ProcessMultipleUrls(IEnumerable<string> links)

      As an example:

      
      Surly.ProcessMultipleUrls(new[] {"google.com", "yahoo.com"});
      // result: google.com - http://sur.ly/o/google.com/AA000014
      //         yahoo.com - http://sur.ly/o/yahoo.com/AA000014
      
    • Still there is a method to process the whole HTML:
      public static string Process(string stringToParse)

      For example:

      
      var processedResult = Surly.Process(htmlToParse);
      // processedResult will contain processed HTML
      

    In addition there are several options:

    • public static string ToolBarId domain toolbar ID;
    • public static bool UseShortUrls enable\disable URL shortener;
    • public static List<string>WhiteList collection of trusted URLs which are not required to be processed. Can be added to the list in the following way:
      Surly.WhiteList.Add("google.com");

    Example usage without web.config

    
    // define toolbar ID
    Surly.ToolBarId  = “AA000014”;
    
    // enable shortener
    Surly.UseShortUrls = true;
    
    // add google.com to the whitelist. No need to process google.com
    Surly.WhiteList.Add(“www.google.com”);
    
    // process several URLs
    var processedResult = Surly.ProcessMultipleUrls(new[] {“www.yahoo.com”, “www.google.com”));
    // processedResult collection will contain processed URLs except www.google.com
    

    Download

  • Cannot find an answer to your question?

    If you cannot find an answer to your question, or you experience some difficulties with Sur.ly integration, or you want to implement Sur.ly in non-PHP project, then contact us at contact@surdotly.com.